The File That Can't Talk

You can go to HuggingFace and download a state-of-the-art AI model today. Not a demo, the real thing; the same kind of model that powers the chatbots in the news. It arrives as one enormous file, roughly the size of 40 two-hour films. And when you open it, nothing happens. It cannot talk, listen, or think. It cannot even run. So what turns that inert file into something you can ask to fix your life at 2 a.m.?

That question is what this series is about. LLM Basics opened the box and looked at the machinery inside a language model, the attention heads and transformer blocks. This series walks in the opposite direction, outward. Between the model file on a disk and the chat bubble on your screen sits half a dozen layers of software that almost nobody explains, and every confusing thing you have heard about AI, thinking mode, tool use, agents, lives in one of those layers.

To keep things concrete, one message will travel with us through the whole series "What's the weather in Paris?" By the final part you will be able to follow it every step of the way, from your keyboard down through the layers and back up. This part starts where the message will eventually arrive i.e. at the model file itself.

A very large file of numbers

A language model is a file (or sometimes a small set of files) containing numbers called weights, the learned dials you may remember from ML Basics. Billions of them. A mid-size open model weighs in somewhere between 17 and 60 GB depending on how it is compressed, and the frontier models are far bigger. That is the whole product. There is no program inside, no rules, no database of facts, and nothing about it "runs". If you have read What's Actually in the Box?, you have already seen the full inventory: a tokenizer, an embedding table, and layer upon layer of learned matrices. Numbers all the way down.

The numbers come from training: the model read an enormous amount of text once, and the weights are what it learned from the experience. By the time you meet a model, training is over. It happened months earlier, in a data centre, at a cost that would make your eyes water. The file you download is frozen.

Frozen is worth sitting with, because it breaks the most common assumption people bring to AI. The model does not learn from your conversations. When you chat with an AI, the weights do not change. Not one number. You can tell it your name a hundred times and the file stays bit-for-bit identical. (Companies may collect conversations to help train future models, which is a real privacy question, but it is a different thing from the model in front of you learning.) How a frozen model still manages to remember your name within a conversation is Part 3's story, and it is stranger than you would guess.

For now, here is the analogy that will carry the first half of this series: the weights are a recipe book. All of the knowledge, sitting inert on a shelf, cooking nothing. The analogy breaks in one honest place: a recipe book can be read by a person, while the weights are billions of numbers that no human can read meaning from directly. But the shelf part is exactly right. A recipe book does not cook, and a model file does not think.

So what does the model do?

Once something or someone makes it go, the model does one thing. Given a piece of text, it predicts what comes next. That is the entire skill. Everything impressive and everything baffling in this series, the chat, the reasoning, the agents, is machinery wrapped around a machine that continues text.

Word-chunks, not words

There is a catch hiding even in that sentence: the model does not see text the way you do. Before anything reaches it, the text is chopped into tokens, word-chunks that each map to a number in the model's fixed dictionary. "Paris" might be one token and "weather" another, while an unusual word gets broken into several pieces. As a rough anchor, a token is about three quarters of an English word, so a page of writing is around 400 tokens. The model reads numbers and produces numbers, start to finish.

This explains one of the internet's favourite party tricks. Ask a model how many R's are in "strawberry" and it may miscount, which seems absurd for something that writes decent poetry. But the model never sees letters. It sees "strawberry" as one or two numbered chunks, and there is no spelling inside a number.

A ranked list, not an answer

The other surprise is that the model does not output "the next token" at all. It outputs a ranked list: every token in its dictionary, scored by how likely it is to come next. Something has to choose from that list, and how boldly to choose is set by a dial called temperature. Turn it low and the top choice wins nearly every time, which reads as careful and predictable. Turn it up and lower-ranked tokens get a real chance, which reads as creative, and occasionally as unhinged.

The demo below is a real ranked list in miniature: a model mid-sentence, deciding how to continue "The capital of France is". Drag the dial and watch the shares shift, then sample some completions and see what temperature does to them.

Two things are worth noticing. At low temperature the model is close to deterministic: the same text in, the same continuation out, nearly every time. And even at high temperature, "croissants" stays rare. Temperature reshapes the ranking, it does not abolish it. Every word an AI has ever said to you was a draw from a list like this one.

Supercharged autocomplete, with an asterisk

You have probably heard the dismissive summary: "it's autocomplete on steroids." As a description of the mechanism, that is fair, and this series will not pretend otherwise. What the phrase hides is what happens at scale. Trained on enough text, a next-word machine is forced to get good at whatever makes next words predictable: grammar, facts, logic, style, the shape of an argument. It ends up completing reasoning-shaped text, not phrases. The mechanism is autocomplete. The behaviour, at this size, feels like something else, and both halves of that sentence are true.

So here is the puzzle this series exists to answer. On the desk sits a frozen file that cannot run, containing a machine that can only continue text, one word-chunk at a time. And yet tonight, somewhere, that same file will appear to hold a conversation, remember a name, check the weather in Paris, and apologise for a mistake. The first step is getting the numbers to compute at all, and that takes a second program, one that almost nobody outside the field can name.

Next: The Kitchen: What Runs the Numbers: the inference engine, why graphics cards, and what those cryptic Q4 and Q8 labels on download pages actually mean.