II · THE IDEA · ARTIFICIAL INTELLIGENCE
Supervised Fine-Tuning
▶ Listen · narrated
The model that finishes your sentence and the one that answers your question are often the same weights, separated by a few hours of training on conversations instead of documents.
At a glance
- What it does
- Adapts a pretrained model to follow instructions by training on example conversations
- Dataset size
- Often 10,000–100,000 prompt-response pairs, orders of magnitude smaller than pretraining
- What changes
- Behaviour and format, not the bulk of factual knowledge
- Common name
- SFT, the first stage in instruction tuning
Imagine you have learned a language by reading thousands of books, but no one ever showed you how conversations work. You can write fluently, but if someone asks you a question, you might write another chapter instead of answering. Supervised fine-tuning is like spending a week watching conversations and learning the pattern: someone asks, you answer. You already knew the language. Now you know the shape of a dialogue. The model has seen trillions of words during pretraining, learning grammar, facts and reasoning. Fine-tuning shows it tens of thousands of examples where text arrives as a question and the correct continuation is a direct answer, not more web text. It is learning a format, not filling in missing knowledge.
Supervised fine-tuning continues training a pretrained model on a dataset of input-output pairs, using the same causal language modelling objective but with loss computed only on the output tokens. The dataset is typically 10⁴ to 10⁵ examples, several orders of magnitude smaller than pretraining corpora, and training runs for a small number of epochs — often one to three — to avoid catastrophic forgetting. The model's weights are initialised from the pretrained checkpoint, and the learning rate is usually lower than during pretraining. The effect is to shift the model's output distribution toward the style and structure demonstrated in the fine-tuning set, without substantially rewriting the representations learned during pretraining. This is why fine-tuned models retain factual knowledge and reasoning ability from the base model while exhibiting markedly different behaviour in terms of format, refusal rates and instruction-following. The format of the fine-tuning examples — including delimiters, system messages and turn structure — becomes part of the learned distribution, which is why inference-time prompts must match the template used during fine-tuning to achieve optimal performance. The technique is sensitive to data quality: a small number of high-quality demonstrations often outperforms a large number of noisy examples.
Look closer
The dataset is curated demonstrations, not raw text
Where pretraining ingests web pages, books and code as they were written, supervised fine-tuning requires pairs: a prompt and the response a human (or another model) wrote for it. Someone had to compose these, label them, or extract them from logs. The InstructGPT work that introduced this approach to a wide audience used around 13,000 such pairs written by contractors. FLAN, a contemporary effort from different authors, used millions of examples generated by converting existing NLP datasets into instruction format. Both worked, but the data looked completely different.
It takes far fewer steps than pretraining
A model pretrained on hundreds of billions of tokens might be fine-tuned on tens of millions — three or four orders of magnitude less data, seen for just a few epochs. The brevity is not a limitation. The model has already learned language, facts and reasoning patterns. Supervised fine-tuning teaches it a new prior: that text arrives as a question or instruction, and the expected continuation is a direct response rather than more internet text. You are adjusting the distribution it samples from, not filling an empty network.
The format of the prompt becomes part of what the model learns
If every demonstration in the fine-tuning set begins "User:" and the response begins "Assistant:", the model learns that this structure predicts helpful behaviour. Omit those markers at inference time and quality drops, because you have moved outside the distribution it was shown. The same applies to system messages, delimiters and turn-taking conventions. They are not decorative metadata; they are part of the input the model was trained to continue from. Change the template and you are, in a small but real way, asking the model to generalise to something new.
The story
A model fresh from pretraining has seen trillions of words, but it has seen them as documents, forum posts, code and web pages. It has learned that text continues in the style and topic it began with. Ask it a question and it may answer, or it may continue with another question in the style of a FAQ page, or it may do something else entirely that fits the pattern of internet text. It is not confused; it is doing exactly what the statistics of its training data suggest.
Supervised fine-tuning intervenes with a small, carefully shaped dataset. Each example is a prompt — a question, an instruction, a request for explanation — followed by a response that demonstrates the behaviour you want. The model trains on these pairs using the same next-token prediction objective as pretraining, but now the distribution has changed. The text no longer meanders through topics or mixes genres. It asks, then answers. It requests, then complies.
The dataset is small by pretraining standards, often tens of thousands of examples rather than billions of documents. Yet the effect is large. After fine-tuning, the model behaves as though conversation is the natural form of text, and helpful responses are what text does. It has not forgotten how to write poetry or complete code, but those capabilities now sit behind a different prior. The model expects to be prompted, and it has learned that the shape of a good continuation is an answer.
The technique itself is not new. Fine-tuning a pretrained model on a smaller task-specific dataset is older than modern language models. What changed in recent years was the realisation that instruction-following could be framed as a task, and that a diverse set of demonstrations — not a single narrow objective — could teach a general capacity to follow instructions. The FLAN work showed that fine-tuning on many different tasks, each phrased as an instruction, improved zero-shot performance on new tasks. The InstructGPT work showed that a relatively small number of high-quality human demonstrations could align a model's behaviour with what users actually wanted, reducing harmful or unhelpful outputs without sacrificing capability.
The training process is standard supervised learning. The model sees a prompt, generates a response, and updates its weights to make the demonstrated response more probable. The loss is computed only on the response tokens, not the prompt, so the model learns what to generate given an instruction, not how to generate instructions themselves. This asymmetry is why fine-tuned models are better at answering than asking, and why they do not spontaneously start prompting you.
The result is a model that feels qualitatively different to use, even though the architecture and the vast majority of the knowledge remain unchanged. The shift is in distribution, not capacity. You have taught it a new genre.
Why it mattered then
Supervised fine-tuning became prominent around 2021 and 2022 as a solution to a specific problem: large language models were powerful but erratic. They could write essays, generate code and answer questions, but they could also ramble, refuse simple requests or produce content that was technically fluent but useless or harmful. The models had capability but no consistent interface. Prompting strategies helped, but they were brittle and required expertise. The InstructGPT paper, published by OpenAI in March 2022, demonstrated that a relatively small dataset of human-written demonstrations could reshape behaviour in a stable, generalisable way. The fine-tuned model was preferred by human evaluators over a model 100 times larger that had only been pretrained. The result was not just better outputs; it was a model that felt like it understood what you wanted. The FLAN work, from Google researchers, showed that instruction tuning on a broad mixture of tasks improved zero-shot performance, meaning the model could handle new instructions it had never seen. These results mattered because they made large models practical. Supervised fine-tuning became the standard first step after pretraining, the bridge between a text-completer and a usable assistant.
Why it matters now
Supervised fine-tuning remains the foundation of every instruction-following model in wide use. It is the first stage of alignment, the step that turns a base model into something a non-expert can interact with. Models released as "instruct" or "chat" variants have all undergone this process, and the quality of the fine-tuning dataset is often the difference between a model that feels helpful and one that feels confused. The technique also matters because it is accessible. Pretraining a large model requires clusters and budgets that few organisations have. Fine-tuning requires far less: thousands of examples, not billions, and hours or days of compute, not months. This has made it possible for researchers, companies and open-source communities to adapt models to specific domains, languages or behaviours without starting from scratch. A base model can be fine-tuned for medical question-answering, legal summarisation or a language underrepresented in the pretraining corpus, using datasets that are large by research standards but small by pretraining standards. It also set the template for what followed. Supervised fine-tuning is usually the first stage in a longer alignment process that includes reinforcement learning from human feedback, but even that later stage depends on a model that has already learned the basics of instruction-following. The demonstrations matter, and they shape the model in ways that are hard to undo later.
The surprising detail
The fine-tuning dataset does not need to cover every possible task or topic. The FLAN experiments showed that training on a diverse mixture of tasks — summarisation, translation, question-answering — improved performance on entirely new tasks the model had never seen in the fine-tuning set. This was surprising because it suggested the model was learning something more abstract than the tasks themselves: a general capacity to recognise and follow instructions. It was not memorising patterns; it was learning a prior about what kind of text follows an imperative sentence.
Remember this
Supervised fine-tuning changes how a model continues text, not what it knows. Behaviour shifts with thousands of examples; knowledge requires billions.
Test yourself
A company fine-tunes a model on 50,000 demonstrations in English. At inference, non-English prompts produce lower-quality responses. Why, and what does this reveal about what fine-tuning actually taught?
Fine-tuning taught the model that instructions arrive in English and responses should match that distribution. The model's pretraining included other languages, so it retains multilingual capability, but the fine-tuning set created a prior that English prompts predict high-quality responses. Non-English prompts are out-of-distribution for the behaviour the model learned during fine-tuning, even though they are not out-of-distribution for the model's underlying language knowledge. This reveals that supervised fine-tuning is teaching a conditional distribution — how to respond given this kind of prompt — not adding facts. If you want strong multilingual instruction-following, the fine-tuning set must include demonstrations in those languages.
Go deeper
- Training language models to follow instructions with human feedback · arXiv · Long Ouyang et al. · 2022-03-04
- Finetuned Language Models Are Zero-Shot Learners · arXiv · Jason Wei et al. · 2021-09-03
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.