Seven parts ago, a message set out: "What's the weather in Paris?" It has since been tokenised, stapled, briefed, rewritten, ticketed, computed, thought about, and acted on. Time to run it through the whole stack in one go, with every layer named. This is the section to screenshot.

Ten steps, no hand-waving

  1. You type "What's the weather in Paris?" and press send.
  2. The app staples it to the entire conversation so far, plus the hidden briefing, plus any notes it keeps about you. The model remembers none of this, so all of it must travel. (Part 3)
  3. The app packs everything into a standard order ticket, listing the tools it is willing to perform. (Part 5, Part 7)
  4. The server receives the ticket and rewrites the conversation into the model's private script format, ending at the assistant's cue. (Part 4)
  5. The engine feeds that text through billions of frozen numbers, producing ranked lists, one token at a time. (Part 1, Part 2)
  6. In thinking mode, the model first writes scratch notes to itself: "User wants current weather. I cannot know this. There is a weather tool." (Part 6)
  7. It emits a structured request, get_weather(Paris), and stops. It has asked. It cannot do. (Part 7)
  8. The app, not the model, calls a weather service, receives "24°C, sunny", and pastes that into the transcript as a new message.
  9. The whole, now-longer conversation goes back through steps 3, 4, and 5 again.
  10. The model writes "It's 24°C and sunny in Paris right now," streamed to your screen one token at a time. Then it stops existing as a running process, until your next message wakes the entire chain again.
you "What's the weather in Paris?" the app staples the transcript + briefing the order ticket standard API request, tools listed the waiter model server — LM Studio, Ollama, the cloud the script rewrite chat template, roles become markers the kitchen inference engine, one token per pass the recipe book the weights, frozen since training the answer streams back up, one token at a time nothing in this picture remembers anything — the conversation lives in the ticket
The whole machine. Every "mode" in this series is a variation in what travels down this stack and what comes back up.

Why any of this matters to you

Not for trivia's sake. Knowing the layers changes four practical things.

Privacy. "Local AI" now means something concrete to you: the entire restaurant, file and engine and waiter, running inside your own machine, with nothing leaving the building. When you use a cloud service instead, it is not that the machine is different, it is that every transcript, briefing and all, travels to someone else's computers and is handled under a policy. You can now name exactly what it is they receive: the whole ticket, every turn.

Cost. You are billed in tokens, and two consequences fall straight out of the layers. Thinking mode means paying for words you may never see, because the scratchpad is generated text like any other. And because the transcript is re-sent every turn, a long conversation costs more per message the longer it runs. The cheap habit follows immediately: new topic, new chat.

Judgement. When an AI product seems to get worse overnight, the weights are the last thing you should suspect. Between you and them sit the briefing, the template, the server's settings, the tool list, and the app's own behaviour, five layers where the product can change without the model changing at all. "The model got dumber" is possible. "Something in the wrapping moved" is, far more often, the story.

Reading the news. Every "an AI did X on its own" story is a story about software that looped a model into doing X, with tools somebody chose to offer and guardrails somebody chose to build, or skipped. You now know to ask which layer acted: the model that asked, the app that executed, or the engineer who wired them together. The question sounds pedantic. It is where all the responsibility lives.

A glossary to keep

term plain meaning
weights the billions of learned numbers that are the model, frozen after training
token a word-chunk, roughly three quarters of a word, the only unit the model reads or writes
temperature the dial that sets how boldly the next token is picked from the ranked list
inference engine the program that does the arithmetic of prediction (e.g. llama.cpp)
quantisation storing the same weights with fewer decimal places, so the file shrinks
context window the maximum text a model can be handed at once, its desk size
transcript the full conversation text, re-sent to the model on every turn
chat template the rewrite rules that flatten a conversation into the model's private script format
system prompt the hidden briefing an app staples above every conversation
model server the "waiter" that loads models, applies the template, runs the engine, and answers apps (LM Studio, llama-server, Ollama)
API an agreed format in which one program asks another to do something
streaming delivering tokens the moment they are generated, which is why the words type themselves
thinking mode a protocol where the model writes a private scratchpad before its visible answer
tool call a structured request the model emits so the app will perform an action for it
agent tool calling in a loop, with guardrails, until the model declares the job done
MCP an emerging standard plug between tools and models, as the standard API is for chat

That is the series. A file that cannot talk, an engine that only does arithmetic, a transcript re-read from scratch on every turn, a rewrite nobody sees, a waiter speaking a universal plug, and a handful of protocols, thinking, tools, loops, layered on top. You can now follow a message all the way down and all the way back, and explain every step at a dinner table without hand-waving. Use it the next time a headline tells you an AI did something on its own.

Want to go deeper? LLM Basics opens the box this series walked around, the transformer itself. Building AI Agents turns Part 7's loop into working code.

← Back to How LLMs Talk · All series