II · THE IDEA · ARTIFICIAL INTELLIGENCE
Emergence: Abilities That Appear Only With Scale
▶ Listen · narrated
Show a large language model worked reasoning steps and it solves multi-step arithmetic better. Show the same steps to a smaller model and it does worse. Size decides whether the help hurts.
At a glance
- The definition
- An ability absent in smaller models and present in larger ones
- How it is seen
- Task accuracy plotted against training compute, on a log axis
- The shape
- Flat at chance for orders of magnitude, then a turn upward
- The counter-case
- Grokking modular addition, where a sharp jump hid gradual change
- Chief caveat
- All-or-nothing scoring can make smooth progress look sudden
A child learning to ride a bicycle wobbles for weeks — foot down, wobble, foot down — and then one afternoon rides the whole street. Suppose you had kept a chart with one column: seconds upright without stopping. It would read 0, 0, 0, then 40. The chart makes the change look instant. But balance, steering and pedalling had been improving the whole time; the chart simply had no way to record partial success.
Something similar happens when researchers plot what language models can do against how large they are. Some skills improve steadily with size. Others sit at the score random guessing would earn, model after model, and then, past a certain size, start working. Wei and colleagues named these emergent abilities, meaning only this: absent in the smaller models, present in the larger ones.
The bicycle worry applies. If a test marks answers right or wrong with nothing in between, a model that is quietly improving earns nothing for its progress until the moment it gets everything right at once — and then its score leaps. Nanda and colleagues found exactly this in a small model trained on clock-face arithmetic. The visible score jumped, but inside, the network had been building the right method gradually all along. The jump was real in the score and not real in the network. Whether the same explains the jumps seen in enormous models is an open question.
Wei et al. (2022) define an emergent ability operationally: one not present in smaller models but present in larger ones. The evidence is a plot of few-shot task performance — accuracy when the model sees only a handful of worked examples in its prompt — against a scale proxy, most often training FLOPs, the total count of arithmetic operations spent in training, which folds together parameter count and dataset size. Scale goes on a log axis. Across a range of benchmarks the curves sit at chance over several orders of magnitude and then rise, with the turning point differing by task and by model family. Elicitation shows the same pattern: chain-of-thought prompting, where the prompt includes worked reasoning steps, improves multi-step reasoning above some scale and can degrade it below, so the sign of the intervention depends on model size.
The main methodological objection concerns discontinuous metrics. Exact-match scoring multiplies per-step correctness: if a task needs k independent correct sub-steps and per-step accuracy p rises smoothly, the exact-match score p^k stays near zero and then climbs steeply — a sharp curve generated by a smooth underlying trend. Continuous alternatives — the log-likelihood the model assigns to the target tokens, edit distance, partial credit — often expose that trend. This does not refute genuine thresholds; it means a sharp curve is not, by itself, evidence of one.
Nanda et al. (2023) supply the clearest mechanistic case study. A one-layer transformer trained on addition modulo 113 shows grokking: training accuracy rises early through memorisation, test accuracy stays poor, then generalisation appears abruptly. Reverse-engineering the trained network reveals a periodic, Fourier-style representation — inputs encoded as sinusoids at a few frequencies — in which modular addition is carried out through rotations. Crucially, the authors define progress measures: scalars computed from weights and activations that quantify how much of this structure is present. These evolve continuously across the interval where test accuracy is flat, and training decomposes into phases in which the memorising solution and the generalising circuit coexist and then trade off.
Limitations worth stating. The grokking result concerns a synthetic algebraic task and a minimal architecture chosen for tractability; it shows that a visible discontinuity can overlie continuous internal change, not that this explains emergence in frontier models. Progress measures of this kind require first understanding the learned algorithm, which is unavailable for large models on natural-language tasks. Conversely, the metric critique does not account for every reported case, and no supplied evidence decides between threshold-in-network and threshold-in-metric explanations at scale. Practical consequences are narrower: small-scale evaluations may not extrapolate for tasks in this class; report continuous metrics alongside exact-match; and treat compute thresholds as task- and family-specific observations, not transferable constants.
Look closer
The flat part is the interesting part
What draws the eye in these plots is the rise, but the informative feature is what comes before it. On many of the tasks surveyed by Wei and colleagues, the smaller models do not score slightly above guessing, nor trend gently upward. They sit at chance — the score you would expect from picking answers at random — across a wide span of training compute. Then, at some size, the curve leaves the floor. Nothing about the training recipe changed at that point. Only the scale did.
Prompting strategies can flip sign
Some abilities are not about the task at all but about the technique used to elicit it. Chain-of-thought prompting, where the model is shown worked reasoning steps before being asked a new question, improves performance on multi-step problems in sufficiently large models. In smaller ones it can do worse than answering directly. So the same instruction is a help or a hindrance depending on scale, which is a stranger result than a task simply becoming easier with size.
A jump in the score, a slope inside the network
Nanda and colleagues trained a small transformer on addition modulo 113 — arithmetic that wraps around after 113, like a clock face — and watched test accuracy stay poor for a long time and then jump. Inside, the story was gradual. The network was progressively assembling a structured, periodic algorithm for the task, and the visible jump came only when that structure was finally good enough to generalise to unseen sums. The internal change was continuous. The metric was not.
The story
Much of the current argument about large language models can be read off a single kind of chart. Along the bottom runs scale — usually training compute, the total amount of calculation spent teaching the model, though sometimes the number of parameters or the size of the training data. The axis is logarithmic, meaning each step to the right multiplies the amount rather than adds to it: 10 times as much, then 100, then 1,000. Up the side runs accuracy on some task. For many tasks the line is a steady slope. More scale buys more skill at a predictable rate, and a small model is a fair preview of a large one. For other tasks the line does something stranger. It lies flat at chance — the score random guessing would earn — model after model, across several of those multiplying steps, and then, past a certain size, it turns and climbs.
Wei and colleagues gave that second shape a name in 2022, and a deliberately modest definition to go with it. An ability is emergent if smaller models do not have it and larger ones do. Nothing more. The definition makes no claim about what happens inside the model, and none about understanding. It is a claim about a curve: where along the scale axis a capability first shows up. The word carries grander associations from other fields, which is exactly why the authors pinned it to something you could plot and check.
The shape matters because the flat part gives no warning. The standard way to forecast a bigger model is to test smaller ones, fit a trend to their scores, and extend the line. On a task with this shape, that method fails in the worst way: every small model scores at chance, the extended line predicts the task is out of reach, and the large model then does it. The models that exist tell you least about the model you are about to build precisely on the tasks where you most want a forecast.
Prompting shows the same size-dependence in a sharper form. Chain-of-thought prompting means showing the model worked examples that spell out the reasoning steps before asking a new question. In sufficiently large models this improves accuracy on multi-step problems. In smaller ones it can make answers worse than asking directly. The same instruction is a help at one size and a hindrance at another — a stranger result than a task simply getting easier as models grow.
Then the complication, and it is serious: a sharp curve can be manufactured by the way you score. Walk through it. Suppose a test demands an exact multi-digit answer and grades all or nothing — anything short of perfect scores zero. Now suppose a model is genuinely improving as it scales: right on more digits, wrong less often at every step. The scoreboard records none of that. Getting most of the answer right pays exactly what getting none of it right pays. So the plotted score sits on the floor while the model's real competence rises smoothly — until the size at which the model gets every part right at once, when the score leaps from nothing to something. The improvement was gradual. The measurement was a cliff. Every claim of emergence has to face the possibility that the jump lives in the scoring rather than in the model.
Can the two be told apart? In one small case, yes, because the model was small enough to open up. In 2023 Nanda and colleagues trained a one-layer transformer — a minimal version of the architecture behind large language models — on a single task: adding two numbers modulo 113, which means adding them and keeping only the remainder after dividing by 113, the way hours wrap around a clock face. During training, accuracy on the examples the model had seen rose early, because it had memorised them. Accuracy on sums it had never seen stayed poor for a long stretch — and then jumped. The pattern is called grokking, and from the outside it looks like understanding arriving all at once.
Inside, nothing arrived all at once. Reverse-engineering the trained network showed what it had built: a periodic representation, one that encodes each number as a set of smooth repeating waves, in which clock-face addition becomes a matter of rotation — turning the hands rather than counting. And it had been building that machinery gradually, across the very stretch when the test score sat still, while the memorised answers and the general method coexisted and slowly traded places. The researchers made the hidden progress visible with progress measures: numbers computed from the network's weights and internal activity that track how much of the structure exists. Those measures rose smoothly while the accuracy curve did nothing. The jump was real in the score. It was not a jump in the network.
That is why the two papers belong in one lesson, and also why neither ends the argument. The grokking result concerns a tiny network on an artificial arithmetic task, chosen because it could be understood completely; building such progress measures requires first knowing what algorithm the network learned, which nobody can yet do for a frontier model on a language task. It proves that a visible jump can sit on top of continuous change, not that it always does. Whether the capability jumps seen in very large models are thresholds in the network, artefacts of the scoring, or a mixture of the two remains open — and the honest chart carries that caveat in its caption.
Why it mattered then
The definition solved a vocabulary problem. Researchers kept noticing that larger models could do things smaller ones could not — follow instructions, work through arithmetic step by step, handle tasks nobody had trained them for — and were describing this in words heavy with implication. Wei and colleagues offered a spare, testable criterion instead: absent in smaller models, present in larger ones. That turned an impression into something you could chart and argue about. It also arrived when scaling had become the field's dominant strategy, which made forecasting the results of scaling a practical necessity. Smooth curves allow planning: test a small model, extend the trend, budget accordingly. But if some capabilities stay invisible until a size threshold is crossed, a small pilot model says little about the large one you are about to train — and safety testing done only on smaller systems has a built-in blind spot, because the behaviour you most need to catch may not exist yet at the size you can afford to test. The grokking analysis mattered for a different reason. It showed that what a network's score does and what the network itself does can come apart, and that the gap closes if you build the right instruments. That is the case for interpretability — the study of what networks compute internally — as a serious research programme: in this instance the interesting change happened exactly where nobody was looking.
Why it matters now
Anyone deciding whether to train or buy a larger model is betting on the shape of a curve. If the capability you need improves smoothly with scale, a small pilot gives a usable forecast. If the task sits at chance until some threshold, the pilot tells you almost nothing, and the honest answer is that you will not know until you try. Working out which kind of task you face is worth more than any single benchmark number. The scoring critique has practical bite too. If your evaluation marks answers exactly right or entirely wrong, you will see cliffs whether or not the model has any. Softer measures — partial credit, or the probability the model assigns to the correct answer rather than only whether it produces it — often reveal steady progress under an apparently flat line. Sometimes the harsh metric is the right one; to a user, a wrong answer is simply wrong. But the choice of metric shapes the curve, and you should know you are choosing. The grokking result generalises as a warning about interpretation. When a model's behaviour changes abruptly, the tempting story is that it has gained something. Sometimes the truer story is that a structure it had been assembling for a long time finally became good enough to register in the score. Telling the two apart means looking inside the network — which is why measurements built from a model's internals are more than an academic exercise.
The surprising detail
The clearest evidence that a sudden jump can hide gradual change comes from a task with no practical use at all: adding two numbers and keeping the remainder after dividing by 113. The network was a one-layer transformer, and its smallness was the point. Because both the task and the model could be understood completely, the researchers worked out exactly what it had built — a periodic representation in which clock-face addition becomes rotation — and then defined quantities that tracked its assembly. Those quantities rose smoothly across the very stretch in which the test score appeared to do nothing. Whether the same holds in models many orders of magnitude larger is exactly what nobody can yet show, because the method requires understanding the learned algorithm first.
What is disputed
The central claim here is disputed and should be held loosely. That some capabilities appear only in larger models is a documented pattern across many tasks; what it means is not settled. The strongest objection is that all-or-nothing scoring can manufacture a sharp curve from smooth underlying improvement, and the grokking analysis of modular addition provides a worked case where a jump in test accuracy sat on top of continuous internal change. That case involves a one-layer transformer on a synthetic arithmetic task, which is a long way from a large language model on natural-language benchmarks; it demonstrates that the metric can mislead, not that it always does. The compute figures at which curves turn upward vary by task and by model family and should not be treated as constants of nature. Nothing in the supplied evidence establishes whether large-model capability jumps reflect thresholds in the network, thresholds in the measurement, or both.
Remember this
Emergence, as this literature defines it, is a claim about a curve: an ability absent in smaller models, present in larger ones. Whether the jump lives in the network or in the scoring is still contested.
Test yourself
A colleague shows you a benchmark where accuracy is zero for every model below a certain size, then rises steeply above it, and concludes that the models have crossed a threshold in capability. What would you want to check before agreeing, and what evidence would settle it?
First, how the benchmark is scored. If credit is all or nothing — an exact multi-digit answer, an exactly matching string — then a model improving steadily earns zero for all of that improvement until every part is right at once, and the cliff may belong to the metric rather than the model. Re-scoring with partial credit, or looking at the probability the model assigns to the correct answer rather than only whether it emits it, will often show a smooth rise under the flat line. Second, what is happening inside. The grokking work on modular addition is the cleanest example available: test accuracy jumped, but measures derived from the network's internals showed the relevant structure being built gradually throughout the flat period. So the evidence that would settle it is a smoother measurement, external or internal, that tracks the same underlying competence. If every such measure also shows a cliff, the threshold claim is stronger. In practice, for very large models on complex tasks, this check is hard to perform and the question is not resolved.
Go deeper
- [2206.07682] Emergent Abilities of Large Language Models · arxiv.org
- [2301.05217] Progress measures for grokking via mechanistic interpretability · arxiv.org
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.