Section 18

Learning vs. memorizing

Why memorizing isn't learning

Driving the training loss to zero is easy if you give the model enough knobs: it can simply memorize the answer to every training example. But a lookup table of the training set is useless: the whole point is to do well on data you’ve never seen. The ability to do that is generalizationgeneralizationHow well a model performs on data it never saw during training. The whole point of pre-training is to generalize, not to memorize the corpus.See in glossary →, and it, not training loss, is the real goal. This section is about the gap between memorizing and learning, and how to stay on the right side of it.

Measure on data you didn’t train on

If you only ever look at the training loss, you can’t tell the difference between a model that learned the pattern and one that memorized the examples: both score well. So you hold data back. Split your dataset into a training set, which the model learns from, and a test set, which it never sees during training and which you use only to check it. This train/test splittrain/test splitHolding back part of your data from training so you can measure the model on examples it never saw. Good results there are evidence that its patterns carry over to similarly collected new data.See in glossary → is the single most important habit in machine learning; without it you are flying blind.

  • Low training error and low error on held-back data: evidence that the model generalized to new examples collected the same way. Success, as long as the held-back examples fairly represent what the model will face.
  • Low training error but high error on held-back data: a generalization gap, often called overfittingoverfittingWhen a model memorizes training-set quirks instead of learning general patterns, so it does well on training data but poorly on new data. Rarely the main worry in single-epoch LLM pre-training, but it shapes data choices.See in glossary →. Memorization is one possible cause. Misleading patterns that happened to match the answers in training, leaked information, or differently collected evaluation examples can create a gap too.
  • High training error: evidence that the current setup has not fit the data. The model may be too weak, but failed training, too much regularizationregularizationA change to training that tries to improve performance on new data, often by limiting how freely the model can fit the training examples. Weight penalties, dropout, and early stopping are common forms.See in glossary →, poor inputs, or incorrect labels can look the same.

See the gap open

The dots below come from a smooth underlying curve (dashed) plus noise. The model is a polynomial, and the degree slider controls how many knobs it has: how wiggly it’s allowed to be. It’s fit to the gold training dots only; the hollow teal test dots are held out. Watch both error numbers as you raise the degree.

Memorizing vs. learning
The curve is fit to the training dots only. The hollow test dots are held out. Raise the degree and watch the test error diverge from the train error.
train error 0.033test error 0.043gap 0.011
underfitting — too rigid to follow the real shape
A high-degree polynomial can thread every training dot exactly — train error near zero — yet swing wildly between them, so it does worse on the held-out test dots. That gap is the signature of memorizing instead of learning. Turning up λ punishes extreme coefficients, smoothing the curve back toward the true trend. The dashed line is the real function the noise is hiding.

At low degree the curve is too stiff to follow the real shape: both errors are high, that’s underfitting. Raise the degree and both errors drop: the model is learning. But past a point, something telling happens: the training error keeps falling toward zero while the test error turns around and climbs. The curve is now contorting itself to pass through every noisy training dot, and in doing so it swings wildly between them, fitting the noise, not the signal. That divergence between training and test error is the visual signature of overfitting.

Keeping models honest: regularization

The second slider, λ, adds regularizationregularizationA change to training that tries to improve performance on new data, often by limiting how freely the model can fit the training examples. Weight penalties, dropout, and early stopping are common forms.See in glossary →: a penalty on large, extreme parameter values, added to the loss. The model now has to balance fitting the data against keeping its knobs modest, which smooths the wild swings and pulls the test error back down, often letting a high-capacity model behave like a right-sized one. Crank λ and watch the wiggly curve relax back toward the true trend.

Ways to improve generalization come in many flavors: penalizing weight sizes (as here), randomly dropping units during training, stopping training early, creating sensible variations of existing examples, or simply feeding the model more representative data. Weight penalties explicitly favor smaller parameter values; the others change how training proceeds or what evidence the model sees. They share a goal rather than one exact mechanism: steer learning toward patterns that hold beyond the training examples.

Overfitting and underfitting are really statements about the relationship between a model and its data. That points at the thing we’ve taken for granted this whole time: the data itself. The next section takes it seriously: how bad or biased data quietly caps what any model can become, and the discipline of holding some data back so you can trust what your numbers are telling you.