The Script Rewrite: How a Chat Becomes One Long Text

Part 3 ended on a promise: the conversation gets pasted into one long text, and the pasting has rules. It is worth knowing those rules, not because you will ever write them, but because this is the layer where AI products most often go quietly wrong, and almost nobody who experiences it knows where to look.

Three speakers, one of them hidden

To the software, a conversation is not a wall of text. It is a list of messages, each tagged with a role, and there are three of them.

The user role is you. The assistant role is the model's own past replies, which, as Part 3 showed, get re-sent to it along with everything else. And above both sits a role you have never seen in any chat window: the system message, a hidden briefing that the app writes before the conversation even starts. Who you are, how to behave, what to refuse, today's date. Every polished chatbot you have ever used had one of these stapled to the top of every conversation you had with it.

The rewrite

Roles are a bookkeeping fiction, though. The engine, remember, eats one long text. So before anything runs, a chat template flattens the role-tagged messages into a single string, with special marker tokens wedged between the turns, the way a script wedges stage directions between lines of dialogue. The markers tell the model where each turn starts, who is speaking, and where turns end.

Here is the same short conversation, rewritten the way two different (made-up, but faithful in spirit) model families would each expect to receive it:

Look at the very end of the string. It stops at the assistant's opening marker and nothing more. That cliff edge is the entire mechanism of a chatbot's "reply": the model is handed a script that cuts off right where the assistant is due to speak, and, being a machine that continues text, it writes the next line of the script. The reply is not a different kind of act from Part 1's autocomplete. It is Part 1's autocomplete, aimed by a stage cue.

The other thing to notice is that the two formats are different, and this is the detail with consequences. Each model family is trained on conversations wrapped in its own marker style, and it comes to depend on those exact markers the way you depend on punctuation. There is no universal format. The template a model was trained with is the template it must be served with.

Why this invisible layer is load-bearing

Serve a model with the wrong template, and the same file of weights that performed brilliantly yesterday starts behaving oddly. It may ramble past the end of its turn, answer its own questions, start writing your next message for you, or become strangely bad at tasks it handled fine before. Nothing about the brain changed. The stage directions were wrong, so the actor lost the plot.

This failure is invisible by design, which is what makes it worth knowing about. The user sees only the front of the stack, so the review writes itself: "the AI got worse." In the world of home-run models, where people wire together their own file, engine, and settings, a large share of "this model is broken" reports are, on inspection, template stories. It is the plumbing that fails without a puddle.

The briefing document

The system message deserves a longer look before we move on, because it explains something you have felt but maybe never placed. Chatbot personalities are not entirely in the weights. A big part of any product's character, its tone, its rules, its refusals, its habit of saying "As an AI...", is a hidden document being silently prepended to your every conversation, exactly as in the demo above.

Why does it keep saying "As an AI..."? Because somewhere above your conversation sits a briefing along the lines of "You are an AI assistant. Be transparent about being an AI." When a chatbot recites its nature or its rules, you are often hearing the system prompt, paraphrased back out through the next-word machinery. It is the one part of the hidden layer that regularly leaks into view.

This also settles a misconception from the misconception list this series is quietly working through: ChatGPT and Claude are not models. They are apps wrapped around models: a model, plus a briefing document, plus a template, plus tools we have not met yet, plus an interface. Two products built on the same weights can feel like different beings, and the difference is this wrapping.

Our Paris message has now been fully dressed for travel: stapled to the transcript (Part 3), topped with a briefing, and rewritten into one long marker-studded string (this part). What remains is the question of who does all this. Some piece of software has to hold the template, talk to the engine, and speak to apps on the other side. That piece has a name, you may already have it installed, and it is the subject of Part 5.

Next: The Waiter: What LM Studio and Ollama Actually Do: model servers, the standard plug that lets any app talk to any model, and the only six lines of code in this series.