Overfitting and Generalisation

Gradient descent stops when the model is "good enough." But good enough at what? Here is the uncomfortable truth: a model that predicts its own training data perfectly can still be useless.

The training data is never the point. We already know the targets for those rows, that is what made them training data. The entire reason for building a model is to make predictions for inputs it has never seen: the next student, the next house, tomorrow's sensor reading. How well a model performs on unseen data is called generalisation, and it is the only score that ultimately matters.

Memorising versus learning

Think of two students preparing for an exam.

The first student works through past papers to understand the method: why each step follows from the last, how to recognise which technique a question calls for. The second student memorises the past papers, every question, every answer, word for word. Quiz them both on last year's paper and the memoriser scores 100%, beating the honest learner. Put a new paper in front of them and the memoriser collapses, while the learner does about as well as they always did.

A machine learning model can be either student, and nothing in the training loss tells you which one you have. A model that has memorised its training data, noise, outliers, quirks and all, is called overfitted. It has fitted itself not just to the underlying pattern but to the random scatter around it, and random scatter does not repeat in new data.

There is an opposite failure, too: a model too simple to capture the pattern at all (imagine forcing a flat horizontal line through clearly rising data). That is underfitting. It does badly everywhere, on training and new data alike, which at least has the virtue of being easy to notice.

See it happen

Remember from Part 3 that adding higher powers of x gives a model more knobs and lets it bend. More knobs means more capacity, and more capacity is exactly what a memoriser needs.

Below, eleven data points (dark dots) were used to fit three models: a straight line (2 knobs), a gentle cubic (4 knobs), and a degree-9 polynomial (10 knobs, almost one per data point). The knobs are the weights on each power of x, and you will see them called coefficients in the demo and in most textbooks: coefficient is simply the older word for a weight. Five more points from the same source were held back and never shown to the models. Fit each model, admire its training score, then reveal the held-back points. And this time the knobs themselves are exposed below the chart: every coefficient arrives set to its fitted value, and you can grab any of them and see what your meddling does to both scores.

Watch the two numbers as you switch models. The straight line has a modest training error and, once revealed, a similar held-back error: it learned the trend. The degree-9 polynomial drives its training error down close to zero, it snakes through nearly every training point, but between and beyond those points it swings wildly, and its held-back error explodes. That gap between "brilliant on training data" and "terrible on new data" is overfitting. The cubic sits in between: a little more flexible than the line, no real gain, because the underlying pattern here really is close to straight.

Now play with the knobs. Nudge the straight line's w1 and the curve tilts politely, the training MSE creeping up as you stray from the fitted value; you could almost re-fit it by hand. Then switch to degree 9 and breathe on w9, a knob whose fitted value is 0.0003, and the curve convulses. That fragility is what 10 delicately counterbalanced knobs look like: enormous coefficients cancelling each other to the fourth decimal place just to thread through eleven training points. A model that learned the trend barely notices a nudged knob; a model that memorised the points falls apart at a touch.

The fix: hold data back

The memorising student was only exposed because we handed them a paper they had not seen. Models are policed the same way. Before training starts, the dataset is split, a common recipe being:

• Training set (say 70–80%): the only data gradient descent ever sees. Weights and biases are fitted to this and nothing else.
• Validation set (say 10–15%): kept aside during training but checked regularly, like mock exams. If training loss keeps falling while validation loss starts rising, the model has begun memorising, and we stop. This is the early stopping rule from Part 5, now with its full explanation, and the demo below shows what it looks like. The validation set also referees choices like the learning rate or how many x², x³ terms to allow.
• Test set (say 10–15%): touched exactly once, at the very end, for the final grade. It has had no influence on any decision, so it is the closest thing we have to truly unseen data.

Only one of them changes the model

There is a sharp line running through that list, and it is the most important thing to understand about the three sets: only the training set ever changes the weights and biases.

Gradient descent runs on training data alone. Predict, measure the loss, compute the gradients, nudge every knob downhill, repeat. That loop, the entire mechanism of the last three parts, only ever touches training rows.

Validation and test data are put through the model too, but the process stops halfway. Predictions are made, the loss is computed, and then… nothing. No gradients, no nudges, no updates. The weights and biases sit exactly as they were before, unchanged, because the point is to observe the model, not to improve it. It is the difference between practising and being examined: the same questions, the same effort, but during the exam nobody corrects you mid-answer.

This is also what makes the whole scheme honest. If validation data updated the weights, the model would have learned from it, and its validation score would be just another training score, telling us nothing new. The measurement only means something as long as the model stays frozen while we take it.

One caveat, and it is the reason validation and test are separate sets. Validation data never touches the weights directly, but you look at the result and act on it, stopping training here, choosing a different learning rate there. The knobs are not being tuned by gradient descent, they are being tuned by you, slowly, through a hundred small decisions. That is a subtler kind of learning, and it is why the final verdict has to come from a test set that has influenced nothing at all.

Early stopping, seen on the training curve

Here is the chart that early stopping reads. A very flexible model (a small neural network of the kind Part 12 builds, with about 120 knobs; for now just think of it as a curve that can bend almost anywhere) was trained by gradient descent on the same eleven points, and after every step its loss was measured twice: on the eleven training points, and on the five held-back points it never saw. The two lines are the training curve from the last part, drawn once for each set. Drag the slider to move through training time and see what the model looked like at that moment.

Read the right-hand panel first. The green training loss falls the whole way and ends at zero: by ten thousand iterations the model threads every training point exactly. The red held-back loss falls with it at first, bottoms out at about 150 iterations, and then climbs, ending three times higher than its best. Everything after the dotted line is the model getting better at its homework and worse at the exam. Early stopping is nothing more than a rule that watches the red line and stops when it turns upward.

Now drag the slider and watch the left-hand panel tell the same story in shapes. Early on the curve is a gentle rise through the middle of the cloud. Around the dotted line it is about as good as it will get. From there on it starts to develop wiggles, bending to pass through individual training dots, and every wiggle it adds carries it further from the red held-back dots sitting between them. That is overfitting happening in front of you, one iteration at a time.

Other ways to fight it

Early stopping is one lever. Three more are used constantly, and all of them follow from the same diagnosis: overfitting is a model with more freedom than its data can pin down.

• More data. Eleven points cannot pin down ten knobs. A thousand points would. The memoriser only wins because there is so little to memorise; give it more, and the noise it is chasing stops repeating, so chasing it stops paying. This is the least clever fix and usually the most effective one.
• Fewer knobs. If the cubic and the straight line generalise as well as the degree-9 polynomial does, use them. A simpler model cannot bend to fit noise because it cannot bend at all. Choosing how flexible a model should be is a decision the validation set referees.
• Regularisation. Look back at the degree-9 polynomial's fitted coefficients in the first demo: values in the hundreds, cancelling each other to the fourth decimal place. Wild wiggles need huge weights. Regularisation adds a small penalty to the loss for every large weight, a complexity tax, so that gradient descent is nudged towards fitting the data with the smallest weights it can get away with. The two common flavours, called L2 (or ridge) and L1 (or lasso), tax the squares and the sizes of the weights respectively, in the same way MSE and MAE score the errors. Either way the model keeps its knobs but loses its appetite for using them to the full, which is often exactly the amount of flexibility you wanted.

Where benchmark scores come from

This is also the answer to something you have almost certainly seen without knowing what you were looking at. Every time a lab releases a model, the announcement comes with a table of scores: 88.7% on this, 71.4% on that, a chart with competitors' bars next to it. Those are test set results.

The analogy holds all the way through. The student revises from past papers (training set), takes mock exams to see how it is going (validation set), and then sits the real exam once (test set). It is that last number, not their revision, that goes on the certificate for everyone else to read. Benchmarks are the model's exam results, published.

What makes them useful is precisely the discipline described above. The questions are held back from training, the model sees them once, and the score is reported. Because the exam papers are standardised and shared across the industry (names like MMLU, GSM8K, and HumanEval are just widely-agreed question sets, on general knowledge, grade-school maths, and programming respectively), everyone's model can sit the same exam and the numbers can be compared.

And the same discipline explains how benchmarks go wrong. Two failures are worth recognising:

• Contamination. These test sets are published on the internet so that others can use them. Which means they can quietly end up inside the next model's training data, scraped along with everything else. A model that trained on the exam paper will ace the exam, and the score means nothing. It is the student who found the answer key, and this is a genuine and recurring problem in the field, not a hypothetical one.
• Teaching to the test. If a lab tunes its choices to push a benchmark number up, run after run, that benchmark has quietly become a validation set, consulted many times and leaking its quirks into the decisions. The published number drifts away from what it claims to measure.

None of which makes benchmarks useless, they are the best comparison tool available. But it does explain the healthy scepticism you see whenever a new score is announced, and why "how did it do on your problem?" is always the better question. A model's exam results, like a student's, tell you something real, and never the whole story.

Key takeaways

A model's true score is measured on data it never trained on: that is generalisation. When training performance is excellent but unseen-data performance is poor, the model has overfitted, and the more knobs a model has relative to its data, the easier memorisation becomes. Splitting data into training, validation, and test sets is how practitioners catch it, early stopping is that split acting as a tripwire during training itself, and more data, simpler models, and regularisation are the standard ways to prevent it.

Keep this idea close, because the next part hinges on it: it turns out that failing to find the perfect minimum of the loss can be a blessing in disguise.

Next: Local vs Global Minima: what happens when the loss surface has hills and valleys, and why ending up at a local minimum is not always a bad thing.