Section 01

The prediction game

Input → model → guess → score

Many systems in modern AI (the model that finishes your sentence, labels your photo, or writes code) are, underneath, trained by the same simple game. The model looks at some input, makes a guess about an answer, and finds out how wrong the guess was. Then it changes itself a little so that next time the guess is closer. Do that a few billion times and you get something that feels intelligent. This explainer is about that loop, built up one piece at a time, starting here with the game itself.

Input, model, prediction

Strip away the jargon and a machine-learning modelneural networkA function built by stacking many simple operations — mostly matrix multiplies with nonlinearities between them — whose behavior is shaped by tuning billions of internal numbers (its parameters) from data.See in glossary → is just a functionfunctionA rule that takes an input and gives back exactly one output — like a machine: put something in, get something out. Often written f(x); for example f(x) = 2x turns 3 into 6. A model is just a (very elaborate) function from input to prediction.See in glossary →: something takes an input and produces a prediction.

  • Input: the size of a house. Prediction: its price.
  • Input: an email. Prediction: spam or not.
  • Input: the text so far. Prediction: what comes next.

The model is whatever sits in the middle turning one into the other. For now, picture it as a box with a few numbered dials on the side. The same input, with the dials in different positions, produces different predictions. Those dials are the learned part of the model: training changes their settings so the predictions improve and get more and more accurate.

The model as a box of dials
An input goes in and a prediction comes out. The same input, with the dials in different positions, gives a different answer — and "training" is just turning the dials until the answers are good.
inputsize = 2,000 ft²the modela function with knobsthe dials · one number eachw₁w₂w₃bpredictionprice ≈ $410k

A wrong guess needs a score

A guess is only useful if you can say how good it was. So we need a single number that measures wrongness: bigger when the prediction is far from the truth, smaller when it’s close, zero when it’s perfect. That number is the lossloss functionA single number measuring how wrong the model's predictions are on a batch of data. Training works by adjusting the model to make this number smaller.See in glossary →, and it’s the compass for everything that follows. We’ll spend a whole section on how to define it well; for now, all that matters is that low loss = good.

Play the game yourself

Here is the smallest possible version. The machine predicts using one dial, w, with the rule prediction = w × input. The gold dots are the real answers we want it to match. Drag w and watch the score. Then hand the job to the machine: “Take one step” lets it feel which way the score is falling and nudge w that way; “Let it learn” repeats that automatically.

One knob, one rule
The machine guesses ŷ = w · x. The dots are the real answers. Turn the one knob w to make the line pass through them — or let the machine adjust it for you.
x (input) →ŷ (guess)
Each red stick is one prediction's error. The score adds up their squares — one number for "how wrong are we overall." "Take one step" measures which way the score falls and nudges w that way. Repeat, and the machine finds the rule on its own. That measure-and-nudge move is the entire idea; the rest of this explainer just makes it bigger.

Notice what just happened. You didn’t tell the machine the right value of w. You gave it examples and a way to score itself, and the “learning” button found the answer by repeatedly measuring and nudging. No rules about houses or prices were ever written down: only predict, score, adjust.

Where we’re going

We’ll build up in a deliberate order. First we’ll place this game on the map of how machines learn and see how raw data becomes numbers a model can read; then a real model with many dials, a proper definition of the score, the calculuscalculusThe math of how things change. The one piece we need is the slope: at any point on a curve, how steeply it is rising or falling, and in which direction. That single idea is what lets us tell which way to turn each dial to lower the loss — no heavy math required.See in glossary → trick for knowing which way is downhill, the algorithm that turns that into learning, what happens when you stack these functions into a network, and finally how the whole thing becomes a Large Language Model (LLM). Each step is small. Let’s start by asking what “learning from examples” even means.