How Neural Networks Learn, from the ground up

A long-form, interactive explainer

Underneath every modern AI system is one deceptively simple idea: a machine that gets better at a task just by looking at examples. No one hand-codes the rules. The machine adjusts a pile of numbers until its guesses stop being wrong. This explainer builds that idea from absolutely nothing.

We start with something concrete — predicting house prices from a straight line — and follow a single thread all the way up: what a prediction is, how we score it, how the machine figures out which way to adjust its numbers, what happens when you stack these adjustable functions into a linked network of them, and how it works out whether each of a million, billion or even trillion numbers should go up or down.

No machine-learning background is assumed, and there is no heavy math. Every term gets defined the first time it appears; hover any underlined word for a tooltip, or visit the glossary any time.

Where this sits: it's the on-ramp to the LLM Pre-training explainer, which assumes exactly the machinery that this one builds from scratch. The last section connects the two directly: a large language model turns out to be this same idea, learning to predict the next piece of text instead of the price of a house.

Start reading →~6–8 hours, 23 sections

Contents

The learning game

  1. 01The prediction game— Input → model → guess → score
  2. 02Kinds of learning— Supervised, unsupervised, self-supervised, reinforcement
  3. 03Turning the world into numbers— Features, encoding, and why inputs get scaled
  4. 04The simplest model— Weights, bias, and why parameters are just knobs
  5. 05Measuring wrongness— How we measure a wrong answer
  6. 06Drawing a boundary— Turning scores into yes/no decisions

How learning happens

  1. 07Which way is downhill?— Slopes and just enough calculus
  2. 08Gradient descent— The loss surface, step size, and getting stuck
  3. 09Tuning the descent— Making the descent faster and smarter
  4. 10Batches, epochs & noise— Why we estimate the gradient from samples

From a line to a network

  1. 11Neural networks— Layers, activations, and why nonlinearity matters
  2. 12Why depth wins— Features built on features
  3. 13Backpropagation— How the model assigns blame for a mistake
  4. 14The training loop— Forward → loss → backward → update → repeat
  5. 15Where training starts— Where the numbers start
  6. 16When gradients vanish— The problem that stalled deep learning, and the fixes
  7. 17Shapes of networks— Matching the architecture to the data

Learning vs. memorizing

  1. 18Learning vs. memorizing— Why memorizing isn't learning
  2. 19Data quality & the three splits— Garbage in, and holding data back
  3. 20Judging a model— Why the loss is not the metric you care about

The bigger picture

  1. 21A short history— From the perceptron to deep learning, and why now
  2. 22Embeddings— Meaning as geometry
  3. 23From this loop to an LLM— The same loop, where the prediction is the next token