Instruction tuning is born

FLAN, T0, and zero-shot generalization

Papers: FLAN — Finetuned Language Models Are Zero-Shot Learners (Wei et al., 2021) · T0 — Multitask Prompted Training (Sanh et al., 2021) · Super-NaturalInstructions (Wang et al., 2022) · Scaling Instruction-Finetuned Language Models (Flan 2022) (Chung et al., 2022)

A pre-trained model has read large curated collections of text, but ask it “Translate bonjour to English” and it might continue with more French rather than obeying you. It may have the relevant knowledge without the habit of treating your text as a command. The first big idea of post-trainingpost-trainingEverything done to a model after pre-training to turn a raw next-token predictor into a useful assistant: supervised fine-tuning, RLHF, and RL from verifiable rewards.See in glossary → is a direct fix: if you want a model that follows instructions, train it on examples of instructions being followed.

Pretrain-then-finetune, generalized

You have already seen the seed of this idea. The GPT-1 recipe was pre-train a language model on raw text, then fine-tune it on a labeled task. That worked, but you got one fine-tuned model per task: a sentiment classifier here, a question-answerer there. Each adaptation was a dead end that only knew its one job.

Instruction tuninginstruction tuningFine-tuning on many tasks phrased as natural-language instructions so the model learns to follow instructions in general — including ones it never saw in training.See in glossary → takes that same fine-tuning machinery and asks a more ambitious question: what if, instead of fine-tuning on one task, we fine-tune on many tasks, each phrased as a natural-language instruction, and then test on tasks the model was never trained on? The technique itself is just supervised fine-tuningsupervised fine-tuning (SFT)Training a pre-trained model on curated (prompt, response) pairs with the ordinary next-token objective, so it imitates demonstrated assistant behavior. The first stage of post-training.See in glossary →: ordinary next-token training on (instruction, response) pairs, nothing exotic. What changes is the framing of the data and, as it turned out, the entire character of the resulting model.

FLAN: instruction tuning unlocks zero-shot

Google’s FLAN (Finetuned Language Net; Wei et al., 2021) was the paper that made the case. The authors took a 137B-parameter pre-trained model and instruction-tuned it on 60+ NLP datasets (translation, summarization, natural-language inference, sentiment, and more), but with a crucial twist: each dataset was rewritten into several natural-language instruction templates. Instead of feeding the model a raw premise–hypothesis pair, they wrote things like “Does the premise entail the hypothesis? Premise: … Hypothesis: …”

Then came the test that mattered. They grouped the tasks into clusters, held an entire cluster out of training, and evaluated the model on it zero-shotzero-shotPerforming a task from instructions alone, with no examples given. GPT-2 showed a pre-trained LM can do many tasks zero-shot, just by being prompted.See in glossary →: no examples, just the instruction. FLAN substantially beat the same model’s plain zero-shot performance, and on many held-out tasks beat even GPT-3’s few-shot results. The headline, captured in the title, was that finetuned language models are zero-shot learners: training to follow many instructions made the model follow instructions in general, including kinds it had never been trained on.

T0: smaller, open, and template-explicit

Almost simultaneously, T0 (Sanh et al., 2021) from BigScience showed the same effect at a fraction of the size (an 11B encoder–decoder model) and did it in the open. The name encodes its lineage: the T is for T5, the text-to-text model it’s built on, and the 0 marks the zero-shot generalization it was designed to reach. T0’s lasting contribution was methodological: it leaned hard on explicit prompt templates, crowd-sourcing many differently-worded prompts per dataset so that a single task was seen through dozens of surface phrasings. A model trained this way can’t latch onto one rigid format; it has to learn the task behind the wording. T0 matched or beat models 16× its size on held-out tasks, and made the recipe reproducible for everyone.

Scaling the recipe: Super-NaturalInstructions and Flan 2022

Once the effect was established, the obvious move was to push every knob. Super-NaturalInstructions (Wang et al., 2022) assembled a benchmark of 1,600+ tasks, each with a declarative instruction, drowning the model in instruction variety and giving the field a hard test of cross-task generalization.

Then Flan 2022 / Flan-T5 (Chung et al., 2022) ran the systematic scaling study. It pulled three levers at once and found they compound:

• More tasks: combining the FLAN, T0, and Super-NaturalInstructions collections into 1,800+ tasks.
• More model: scaling up to 540B parameters (Flan-PaLM).
• More reasoning data: adding chain-of-thoughtchain-of-thought (CoT)Having a model write out intermediate reasoning steps before its final answer. Improves accuracy on multi-step problems and is the substrate reasoning RL optimizes.See in glossary → examples, where the instruction’s answer shows its work step by step.

That last ingredient is important. Mixing in CoT data improved performance on many reasoning-sensitive evaluations and helped models produce step-by-step rationales when prompted. Flan-T5 became a widely used open instruction-tuned model, illustrating the value of instruction diversity and task mixture.