LLM Basics
How does a machine actually read, remember, and write? This series tells the story in plain language — starting from the simplest neural network there is, through the networks that learned to read and remember, to attention and the transformer architecture behind ChatGPT. Every part has an interactive demo you can play with, and the maths stays tucked away in optional asides.
New to machine learning? The ML Basics series covers the foundations this one builds on — but you can also start here and follow the links back when you're curious.
-
Part 1
Feedforward Neural Networks
The simplest complete neural network — neurons arranged in layers, data flowing one way — plus the forward pass worked by hand, and the tiny puzzle that once nearly killed the whole field.
-
Part 2
Reading in Order: Networks with Memory
Why the meaning of a word depends on the words before it, and how recurrent neural networks gave machines a notepad — the hidden state — so they could read one word at a time without forgetting.
-
Part 3
The Forgetting Problem
Why simple recurrent networks lose the plot on long text — the vanishing and exploding memory problem — and how the LSTM fixed it with a conveyor belt and three gates.
-
Part 4
Squeezing a Sentence Through a Straw: Encoder–Decoder
How two networks — one that reads, one that writes — made machine translation work, why the fixed-size summary between them became a bottleneck, and what teacher forcing means in plain words.
-
Part 5
Attention: Reading with a Highlighter
The idea that changed everything — let the writer look back at every word of the input and weigh what matters right now, instead of relying on one squeezed summary.
-
Part 6
The Transformer: Attention Is All You Need
How a 2017 paper threw away the reading loop entirely — self-attention, queries, keys and values, multiple heads, and position stamps — and built the architecture inside ChatGPT.
-
Part 7
GPT, BERT, and the Three Flavours of Transformer
Why chatbots only ever look backwards, why search engines look both ways, and how one architecture split into a reader, a writer, and a translator.
-
Part 8
What's Actually in the Box?
When a company ships a language model, what is the thing — a database, a rulebook, a search index? An inventory of what a trained model contains, and a scale to weigh GPT-3 on.