Six parts of machinery: notepads, gates, straws, highlighters, towers, masks. Time for the question nontechnical audiences ask most, and deserve a straight answer to: when a lab "releases a model", what is the actual thing? Is it a database of facts? A gigantic rulebook? The internet, zipped?
None of those. A trained language model is, quite literally, a very large file of numbers, the weights and biases from ML Basics, billions of them, every one discovered by gradient descent. There is no stored text inside, no list of facts, no rules written by anyone. Open the box and here is the complete inventory.
The inventory
1. The tokenizer. The dictionary that chops text into tokens and numbers them, "cat" ↔ 2043. A lookup table, fixed before training even starts.
2. The embedding table. One row of learned numbers per token in the dictionary: row 2043 is the meaning of "cat", as far as the model is concerned, a point in space that training dragged close to "kitten" and far from "carburettor". For a model with 30,000 tokens and 768 numbers each, this table alone is 23 million numbers.
3. The position stamps. The "I am word 7" patterns from Part 5, either a fixed wave recipe or a learned table.
4. The blocks, repeated N times. Each transformer block carries its own set of learned matrices: the Q, K, and V transformations for every attention head (the question-asking, label-making, and content-handing roles from Part 5), a mixer that stitches the heads' reports back together, the feed-forward network where each word does its private thinking, and a few stabiliser dials for the normalisation plumbing. This is where most of the numbers live, and, as far as anyone can tell, where most of the knowledge lives, smeared across the weights rather than filed anywhere.
5. The output layer. The final transformation that turns the last block's numbers into a score for every token in the dictionary, the ranking your next word is sampled from. (In a neat bit of thrift, many models reuse the embedding table for this, flipped on its side.)
That is the whole box. Note what is not in it: no sentences, no documents, no quotes, no index to look things up in. When a model tells you Paris is the capital of France, it is not retrieving a record, it is producing the next token that its weights make most likely. That is also the honest one-line explanation of why models sometimes state falsehoods with total confidence: there is no fact-file to check against, only likelihoods. Remembering and predicting are, inside this box, the same operation.
Weigh a model yourself
The calculator below builds a decoder-only transformer from three dials and counts what goes in the box. The presets are real: BERT-base, GPT-2, and GPT-3.
A few things worth trying. Load BERT-base and look at the embedding table line: roughly 23 million numbers, about 92 MB, a whole vocabulary's worth of meaning, smaller than a phone update. Then load GPT-3 and watch the balance flip: the dictionary barely grows, but depth and width take the total to 175 billion, a third of a tonne of gigabytes. Now drag blocks and numbers per token separately: total size grows steadily with depth but quadratically with width, double the numbers per token and each block quadruples. That is the arithmetic behind the scaling era: most of a modern model is not its vocabulary, it is layer upon layer of Q, K, V, and feed-forward knobs.
The whole story, in one paragraph
You have now walked the entire road. It began with the simplest network of all: neurons in one-way layers, the feedforward network (Part 1). Networks then learned to read in order by carrying a notepad (Part 2), which faded, so gates gave memory a conveyor belt (Part 3). Reading and writing were split into two towers passing a single squeezed summary (Part 4), which choked on long text, until attention let the writer glance back at everything, weighted by relevance (Part 5). Attention proved so capable that the reading loop was deleted entirely, leaving self-attention, position stamps, stacked blocks, and inside every block, the humble feedforward network from Part 1: the transformer (Part 6). Keep only the forward-facing half and it writes; keep the both-ways half and it understands (Part 7). And all of it, every gate, glance, and glossary entry, is nothing but numbers in a file, found by the same recipe from ML Basics: guess, measure the error, roll downhill, repeat a few trillion times (Part 8).
One final connection: a freshly trained box of numbers is a base model, a pure next-token predictor, and as discussed at the end of ML Basics, the step that turns that raw predictor into a helpful assistant is alignment. Architecture builds the instrument; alignment teaches it what to play.
Congratulations, you can now explain, at a dinner table, without a single equation, how the thing that finishes your sentences actually works. Use this power wisely, or at least entertainingly.