II · THE IDEA · ARTIFICIAL INTELLIGENCE
The Emergent Abilities Argument
▶ Listen · narrated
A model trained on twice the data abruptly solves problems the smaller version could not touch. The question is whether something genuinely new switched on, or whether we measured it wrong.
At a glance
- Original claim
- Certain abilities appear sharply and unpredictably above a threshold scale
- Examples cited
- Multi-step arithmetic, word manipulation tasks, some reasoning benchmarks
- Counter-argument
- Discontinuity may be an artefact of nonlinear or discontinuous scoring metrics
- Core dispute
- Whether the model changes qualitatively or the measurement does
Imagine you are teaching a child to spell. At first they get one or two letters right in a five-letter word. Then, after more practice, they get four letters right most of the time. If you only count a word as correct when every letter is perfect, it looks like they learned nothing for weeks and then suddenly mastered spelling overnight. If you count how many letters they get right on average, you see steady improvement the whole time. The debate over emergent abilities is exactly this. Some researchers see large language models suddenly passing tests they failed before and conclude that something new switched on inside the model. Others argue that the model was improving gradually all along, but the test was set up to hide that progress until performance crossed a threshold. Both groups agree on what the models actually did. They disagree on whether the sharp jump in the graph tells us something about the model or something about the graph.
Emergent abilities, as defined by Wei et al., are capabilities not present in small models but present in large ones, where the transition appears sharp and unpredictable when plotted against scale. The original work identified tasks — including multi-step arithmetic, logical inference, and word manipulation — where exact-match accuracy remained near random baseline across models of increasing size, then rose steeply past a threshold measured in parameters or training compute. The authors framed this as evidence of a phase transition, borrowing terminology from statistical physics.
Schaeffer et al. challenged this interpretation by re-evaluating the same tasks using alternative metrics. They replaced exact-match accuracy with token edit distance, per-token cross-entropy, and Brier score, all of which assign partial credit rather than binary outcomes. Under these metrics, performance scaled smoothly and predictably as a function of model size and training compute, following the power-law relationships typical of neural scaling laws. The discontinuity disappeared. Their argument was not that the models performed identically, but that exact-match accuracy is a nonlinear and sometimes discontinuous function of the model's true capabilities, and that this nonlinearity creates apparent emergence where none exists in the underlying system.
The choice between these framings has methodological consequences. Exact-match accuracy reflects deployment reality: many tasks require correct answers, and near-misses fail. But it also conflates two sources of variation: changes in the model's learned representations and changes in how those representations cross a scoring threshold. Smooth metrics reveal incremental learning but may not correspond to task utility. The debate remains open in part because both metrics capture something real, and the question of which one reveals the model's "true" behaviour assumes a fact not in evidence.
Look closer
The original observation was striking
Wei and colleagues documented tasks where performance stayed near zero across multiple doublings of model size, then jumped abruptly when scale crossed some threshold. The curve did not look like a smooth climb. It looked like a switch. The tasks included multi-digit addition, logical inference problems, and certain linguistic manipulations — things that seemed to require a kind of reasoning absent in smaller models. The term "emergent" was borrowed from physics, where it describes properties that arise in a system but cannot be straightforwardly traced to individual components.
The rebuttal focused on how success was scored
Schaeffer and colleagues re-analysed the same tasks using smooth metrics instead of exact-match accuracy. Where the original work counted an answer as correct only if every token matched the target, the rebuttal used measures like token edit distance or the probability the model assigned to the correct answer. Under those metrics, performance improved gradually and predictably with scale. The discontinuity vanished. The argument was not that the models performed identically, but that the appearance of a sudden jump depended on choosing a metric that amplified small improvements into a binary outcome.
The choice of metric is not neutral
Exact-match scoring reflects how we often use models in practice: the answer is either right or it is not. A calculation that returns 487 when the correct answer is 489 fails the task, even though it came close. Smooth metrics, by contrast, give credit for partial progress and reveal incremental learning. Both are defensible. The question is whether the sharp threshold tells us something about the model's internal organisation, or whether it tells us that we built a step function into the evaluation and then observed a step.
The story
In 2022, researchers at Google and several universities published a survey of what they called emergent abilities in large language models. The paper catalogued tasks where performance remained near random chance across smaller models, then rose sharply once training scale — measured in parameters, data, or compute — crossed a threshold. The tasks were not trivial. They included multi-step arithmetic, logical deduction, and problems requiring the model to manipulate words according to abstract rules.
The term "emergence" carries weight. In physics and complexity theory, it describes phenomena that appear at one level of organisation but cannot be straightforwardly predicted from the rules governing the components. The claim was not merely that larger models performed better, which is unsurprising, but that certain capabilities switched on discontinuously. Below the threshold, the model showed no sign of the ability. Above it, the ability was present. The paper presented this as evidence that scale unlocked qualitatively new behaviour.
A year later, a different group published a rebuttal. Schaeffer and colleagues argued that the discontinuity was not in the model but in the measurement. They took the same tasks and re-evaluated them using smooth metrics rather than exact-match accuracy. Where the original work scored an answer as correct only if every token matched perfectly, the rebuttal used token edit distance or the probability mass the model assigned to correct answers. Under those metrics, performance improved steadily and predictably with scale. The sharp jump disappeared. The models were learning incrementally all along; the binary scoring function had hidden it.
The disagreement is not over the data. Both groups agree on what the models produced at each scale. The dispute is over what counts as evidence of emergence. Exact-match scoring reflects real-world use: a miscalculated sum is wrong, not nearly right. But it also creates a threshold effect by design. A model that moves from 5% token accuracy to 95% token accuracy on a multi-step problem will move from 0% exact-match success to near-total success, even though the underlying improvement was gradual. The question is whether the sharp curve reveals a phase transition in the model's capabilities, or whether it reveals that we chose a metric with a built-in step function.
Why it mattered then
The original paper arrived at a moment when scaling laws had become the organising principle of language model research. Larger models reliably outperformed smaller ones, but the improvements had looked smooth and predictable. The claim that certain abilities switched on discontinuously suggested that scale was not merely buying incremental gains but unlocking qualitatively different behaviour. It gave theoretical weight to the industry's scaling agenda and implied that further doublings might produce capabilities no one had yet anticipated. The tasks cited — multi-step reasoning, symbolic manipulation — were exactly the kinds of problems that had long seemed to require something more than statistical pattern matching.
Why it matters now
The debate remains unresolved and continues to shape how researchers interpret new results. When a larger model suddenly passes a benchmark its predecessor failed, the question of whether something genuinely new emerged or whether the test was poorly calibrated matters for forecasting, for safety work, and for resource allocation. If emergence is real, it implies that capabilities may appear unpredictably, which complicates efforts to anticipate risks. If it is a measurement artefact, it suggests we need better benchmarks that reveal incremental progress rather than hiding it. The argument also touches a broader methodological point: the metrics we choose determine the phenomena we observe, and a discontinuity in a graph is not always a discontinuity in the system.
The surprising detail
The rebuttal did not dispute that larger models performed better. It disputed the shape of the performance curve, and therefore the interpretation. This is unusual in machine learning disputes, which more often argue over whether an effect exists at all. Here both sides agreed on the measurements; they disagreed on whether a sharp rise in exact-match accuracy, when underlying token-level accuracy rose smoothly, constituted evidence of emergence or evidence of metric choice. The original authors have not retracted their claim, and subsequent work has continued to cite both papers, often in the same paragraph, as representing two defensible readings of the same data.
What is disputed
The dispute is not settled. Wei and colleagues maintain that emergence is a real phenomenon, pointing to tasks where even smooth metrics show non-monotonic or threshold-like behaviour. Schaeffer and colleagues argue that in every case they examined, the discontinuity vanished under appropriate re-scaling. Both papers have been widely cited, and subsequent empirical work has found examples that support each interpretation depending on the task and metric chosen.
Remember this
A capability that appears to switch on abruptly may be emerging in the model, or it may be emerging in the metric. The data alone does not decide.
Test yourself
You are evaluating a new model on a task where the correct answer is a five-digit number. The small model gets an average of one digit correct per answer. The large model gets an average of 4.9 digits correct. Under exact-match scoring, what does the performance curve look like, and what does it look like under average digit accuracy?
Under exact-match scoring, the small model scores near 0% — getting one out of five digits correct almost never produces the exact right number. The large model still scores near 0%, because 4.9 digits correct means it is nearly always wrong on at least one digit. The curve stays flat. Under average digit accuracy, the small model scores 20% and the large model scores 98%. The curve rises steeply and smoothly. This is the core of the mirage argument: the same underlying improvement in the model's behaviour produces either a flat line or a steep rise depending solely on how you score it. Exact-match accuracy can hide smooth progress and create the illusion of a sudden jump that never occurred in the model itself.
Go deeper
- Emergent Abilities of Large Language Models · arXiv · Jason Wei et al. · 2022-06-15
- Are Emergent Abilities of Large Language Models a Mirage? · arXiv · Rylan Schaeffer et al. · 2023-04-28
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.