II · THE IDEA · ARTIFICIAL INTELLIGENCE

Overfitting and Generalisation

Foundations · Learning the pattern, not the examples · Capacity versus generalisation

▶ Listen · narrated

You can train a model to perfect accuracy on a dataset and watch it fail completely on examples it has never seen. That gap is the central problem.

At a glance

Overfitting: When a model learns the training data so well it fails to generalise to new examples
Classic remedy: Limit model capacity, stop training early, add regularisation penalties
Recent complication: Very large models sometimes generalise better despite having vastly more parameters than training points
Zhang et al. finding: Neural networks can fit completely random labels, yet still generalise when trained on real data

Imagine you are learning to recognise birds from photographs. If you memorise each photograph individually, you will recognise those exact images perfectly but fail on new photos of the same species. That is overfitting. The goal is to learn the features that distinguish species — beak shape, plumage pattern — so you can recognise birds you have never seen before. A model faces the same problem. It can always memorise the training examples if it has enough capacity, but memorisation does not transfer. Generalisation means extracting the pattern that recurs across examples. For decades, the advice was to use a small model and stop training early, because large models memorise too easily. But recent work has shown that very large models, trained until they fit the training data perfectly, can generalise better than medium-sized models stopped early. The explanation is not yet complete, but it seems that when a model has vastly more capacity than it needs, the training process finds solutions that are smooth and regular rather than jagged and memorised. This is why the largest language models work: they are trained in a regime where classical theory predicted failure, but empirical results show success.

Overfitting occurs when a model achieves low training error but high test error, indicating it has learned the training set without extracting the underlying distribution. Classical learning theory, grounded in VC dimension and structural risk minimisation, predicts a bias-variance tradeoff: models with more capacity than training examples will overfit unless regularised. The empirical risk minimisation framework assumes a U-shaped test error curve, rising monotonically past the interpolation threshold where the model can fit the training data perfectly.

Zhang et al. (2017) demonstrated that standard architectures — ResNets, AlexNet — can fit training sets with completely randomised labels to zero training error, achieving perfect accuracy on noise. This shows that the capacity to memorise is not a bug but a feature of neural network expressiveness. Generalisation cannot be explained by capacity constraints alone, because the capacity to memorise is always present. Instead, something about the training process on structured data guides the model toward solutions that generalise.

Nakkiran et al. (2019) documented double descent: test error exhibits two descents as model size increases. The first descent follows classical theory. At the interpolation threshold, test error peaks. But past that threshold, in the overparameterised regime, test error descends again. Models with vastly more parameters than training points, trained to zero training loss, can generalise better than models in the classical regime. The phenomenon has been observed across vision and language tasks, and it appears in multiple dimensions: model size, training time, and dataset size all exhibit double descent under certain conditions.

The theoretical explanations remain incomplete. Implicit regularisation — the tendency of gradient descent to find minimum-norm solutions in overparameterised settings — is one candidate. The loss surface geometry in high dimensions may have many global minima, with gradient descent biased toward flatter, more robust ones. Another line of work points to the neural tangent kernel regime, where overparameterised networks behave like kernel methods with specific inductive biases. What is clear is that classical PAC learning bounds, which predict generalisation error scaling with model complexity, do not tightly describe the overparameterised regime. Large language models are trained precisely in this regime: billions of parameters, datasets smaller than the parameter count in token terms, optimised to low training loss. The empirical success of this recipe preceded the theory, and the theory is still catching up.

Look closer

The classic story is a U-shaped curve
On the horizontal axis, model complexity — more parameters, deeper architecture, longer training. On the vertical, error measured on data the model has never seen. The curve drops as the model learns, then rises again as it begins to memorise training noise rather than extract the underlying pattern. The minimum of that U is where you are supposed to stop. This is the textbook account, taught for decades, and it is not wrong for small models on small datasets.
Zhang and colleagues showed memorisation is always possible
They took standard image classification networks and trained them on datasets where the labels had been randomised — each image assigned to a category at random, with no pattern to learn. The networks fit the training set to perfect accuracy anyway. This was not a failure of the architecture. It demonstrated that the capacity to memorise is always present, which means generalisation cannot be explained by capacity limits alone. Something else in the training process, or the data itself, must be guiding the model toward patterns rather than rote storage.
Double descent breaks the U
Nakkiran and colleagues documented a phenomenon where the test error curve has two descents, not one. The first follows the classic story: error drops, then rises as the model overfits. But if you keep increasing capacity past a critical threshold — roughly where the model has enough parameters to perfectly fit the training set — error drops again. Very large models, trained to zero training loss, can generalise better than moderately-sized models stopped early. The regime is now called interpolation, because the model passes exactly through every training point yet still performs well on new data. Classical learning theory did not predict this, and the explanations remain partial.

The story

Overfitting is what happens when a model learns the training examples so thoroughly that it fails on anything new. It has memorised the data rather than extracted the pattern. The danger is always present, because a neural network with enough parameters can fit arbitrary labels to arbitrary inputs — there is always a set of weights that will reproduce the training set exactly, even if the labels are random noise.

For decades, the standard advice was to limit capacity. Use fewer parameters than you have training examples. Stop training before the model has seen the data too many times. Add penalties that discourage large weights. The mental model was a U-shaped curve: as model complexity increases, test error first drops as the model learns, then rises again as it starts memorising noise. The bottom of the U is the sweet spot, and everything in classical learning theory pointed toward finding it and stopping there.

In 2017, Zhang and colleagues published a result that troubled that picture. They trained standard image classification networks on datasets where the labels had been completely randomised. There was no pattern to learn — each image was assigned to a category by coin flip. The networks fit the training data to perfect accuracy anyway. They could memorise tens of thousands of arbitrary associations. This was not a defect. It was a demonstration that the capacity to memorise is always present, which means something other than capacity must explain why models generalise when trained on real data.

The implication is that generalisation is not about preventing the model from memorising. It is about guiding it toward patterns that compress well, that recur across examples, that reflect structure in the world rather than noise in the sample. The data itself, and the way gradient descent navigates the space of possible solutions, seem to favour certain kinds of functions over others — but the theoretical account of why remains incomplete.

Then, in 2019, Nakkiran and colleagues documented something stranger still. They called it double descent. The classic U-shaped curve appears as expected when models are small. But if you keep increasing capacity past the point where the model can perfectly fit the training set — the interpolation threshold — test error drops again. Very large models, trained to zero training loss, generalise better than moderately-sized models stopped early to avoid overfitting.

The second descent is not universal. It depends on the dataset, the architecture, the optimiser. But it has been observed across image classification, language modelling, and other domains. It suggests that the regime where models have vastly more parameters than training points is not a regime of inevitable overfitting. Something about the way overparameterised networks are trained leads them toward solutions that interpolate the training data smoothly, without the wild oscillations that characterise memorisation in smaller models.

The explanations are still partial. One line of argument points to implicit regularisation: gradient descent, especially in overparameterised settings, tends to find solutions with certain mathematical properties — low norm, low complexity in some measure — even without explicit penalties. Another points to the geometry of the loss surface, which may have many perfect solutions but favour some over others in ways that correlate with generalisation. A third notes that very large models may simply have more ways to represent the same function, and the training process samples from that space in a way that biases toward simpler, more robust versions.

What is clear is that the classical story — more capacity means more overfitting, so use the smallest model that fits — is not wrong, but it is incomplete. It describes one regime well. It does not describe the regime where most large language models are trained.

Why it mattered then

The question of generalisation is as old as statistical learning itself. The classical framework, developed over decades and formalised in the 1990s, treated it as a problem of balancing model complexity against the information available in the training set. The theory predicted that models with more parameters than training examples would overfit catastrophically, and empirical work on small datasets and shallow networks confirmed it. Early stopping, weight decay, and architectural constraints were all designed to keep models in the underparameterised regime, where capacity is the limiting factor. The theory was elegant, the predictions were testable, and the advice was clear: use the smallest model that achieves acceptable training performance, then stop before it memorises.

Why it matters now

The resurgence of neural networks in the 2010s brought models that violated every classical rule and worked anyway. Networks with millions of parameters trained on thousands of examples. Models trained to zero training loss that still generalised. The double descent paper arrived in 2019, after GPT-2 and BERT had already demonstrated that very large models, trained in the interpolation regime, could achieve state-of-the-art results across language tasks. The empirical success preceded the theory. Today, nearly every frontier model is overparameterised by classical standards, and the question is no longer whether to train in that regime but how to do it reliably. The theoretical gap remains. We have recipes that work — large models, large datasets, careful optimisation — but the formal account of why they generalise is still being written. The Zhang result is a reminder that memorisation is always possible. The double descent result is a reminder that it is not always what happens. The gap between those two facts is where the field now works.

The surprising detail

Zhang and colleagues did not just randomise labels. They also randomised the images themselves, replacing real photographs with pure noise, and the networks still fit the training set perfectly. A model can memorise static, which means the ability to generalise to structured data is not a property of the architecture alone. It is a property of the interaction between the architecture, the optimiser, and the statistical regularities in the data. When those regularities are absent, the model memorises. When they are present, something in the training process latches onto them, even though the capacity to memorise is still there, unused.

What is disputed

The mechanisms behind double descent and generalisation in overparameterised networks remain an active research question. Multiple explanations have been proposed — implicit regularisation, loss surface geometry, the lottery ticket hypothesis — but no single account is universally accepted. The phenomenon itself is well-documented across multiple domains, but the conditions under which it occurs reliably, and the formal theory that predicts it, are still being developed. What is clear is that classical learning theory, which predicts monotonic increases in test error past the interpolation threshold, does not describe the regime where large models are trained.

Remember this

Generalisation is not about lacking the capacity to memorise. It is about finding patterns that compress better than rote storage, and very large models sometimes do that more reliably than small ones.

Test yourself

You train two models on the same dataset. One has just enough parameters to fit the training data with a small amount of error. The other has a hundred times more parameters and fits the training data perfectly. Classical theory predicts the second model should generalise worse. Under what conditions might it generalise better, and why does that matter for how we train large language models?

Go deeper

Image: Original diagram, The Daily Triptych. Licence: Original work. Source.

← Back to day 12