Part 5 ended with a processor built out of gates: registers, a clock, and a loop that fetches an instruction, decodes it, executes it, and moves on. Nothing in that description said whether the processor was the one in your laptop or the one on the graphics card next to it, and that is the point. They are the same machine. Both run the loop. Both have a program counter, a decoder, multiplexers, adders, memory cells with word lines and bit lines. If you opened either one down to the transistors you would find nothing in this series that does not apply.

Part 5 told that story with a clerk and a calculator. This part drops the picture and uses the names engineers use. The front end is the part of a processor that fetches and decodes instructions. The execution units are the adders, multipliers and comparators that carry them out. The registers hold the numbers being worked on, and memory holds everything else.

The four parts of any processor, by their real names. The front end fetches the next instruction from memory and decodes it into two decisions, which operation and which registers, shown as the thin blue control lines. The execution units do the arithmetic on numbers taken from the registers and put the result back there. The registers hold the few dozen numbers in use right now, a tick away. Memory holds the program and all the rest of the data, hundreds of ticks away, and talks to the registers through loads and stores. A CPU and a GPU are both exactly this, with the parts in very different proportions.

So what is the difference? It starts with what each machine is for. A CPU is built to run any program at all, and most programs are long chains of decisions in which each step depends on the one before it. A GPU is built for a narrower kind of work, the kind where the same operation has to be applied to millions of different pieces of data that do not depend on each other: every pixel of a frame, every element of a matrix, every weight in a neural network. Once you know the work has that shape you can build a specialised machine for it instead of a general one, and this part is about the three places where the specialisation shows. How many execution units does one front end feed? What is built around the loop? And what does the memory look like? Get these three straight and the rest of the series, which is mostly about how Nvidia's chips answer them, reads easily. The next part is about the wall that both designs have to climb, and the part after that puts them in a race.

The CPU: one instruction, one result

Start with the CPU, because it is the design Part 5 was quietly describing. A CPU core is one front end feeding one small set of execution units, so that each instruction produces one result, and nearly everything Intel, AMD and Arm have done since the 1980s has been to make that single stream of instructions go faster. The last section of Part 5 listed the tricks: a pipeline a dozen or more stages deep, several instructions fetched and decoded per tick, instructions run out of order as their inputs become ready, and a branch predictor that guesses which way every jump will go so that the fetching never has to stop. Each trick is expensive. Between them they take up most of the core, and the arithmetic sits in the middle as a small island, a few integer units, a few floating-point units, and one wide unit that can do sixteen multiply-adds at once when the program asks for it.

The island and what surrounds it. Left, a CPU core drawn as a map: the branch predictor, the fetch and decode stages, the out-of-order machinery, the queues and the caches all exist to keep one small green block busy. Right, that block magnified: a few integer units, a few floating-point units, and the one wide unit, drawn as sixteen lanes that each do a multiply-add whenever the program issues a wide instruction. Areas are indicative, not measured.

That is not a criticism. It is what a general-purpose machine has to be. The programs a CPU runs, a browser, a compiler, the operating system, a game's rules, are long chains of dependent decisions where the next instruction depends on the last result, and there is often nothing else to do while waiting for it. The only way to make a chain faster is to make each link faster, and that is what all that machinery is for.

The other kind of work

Now look at what a graphics card was built to do. A frame at 4K resolution is 8.3 million pixels, and to draw it the same shading routine is run for every one of them: same instructions, different position, different colour, different angle to the light, and no pixel needs to know the answer for any other pixel. Sixty times a second. The main layer of a neural network multiplies a matrix of weights by a matrix of inputs, which is nothing but the running calculation of this series, y = w × x + b, done billions of times with a different w and x each time and the results added up. Simulating a fluid, resizing a photo, decoding video, and a large share of scientific computing all have the same shape: an enormous list of items, one small routine to apply to each, and no item waiting on another.

The work Its shape Suited to
Compiling a program, running the operating system, a web browser, a game's rules, a database query long chains of dependent steps with a decision at nearly every one the CPU
Drawing a frame, multiplying matrices, training and running a neural network, video decoding, image filters, physics simulation the same short routine over millions of items that do not depend on each other the GPU

The rule of thumb is this. If the job can be written as "for every item in this list, do this, and no item depends on another", it is GPU-shaped. If the next step depends on the last result, it is CPU-shaped. Most real programs are a mixture, which is why every computer has both: the CPU runs the program and hands the GPU the list-shaped parts, and Part 12 is about that handover.

Work of the second shape does not need a fast front end. It needs many execution units, and it needs them kept fed. Every trick in the CPU core above is wasted on it. There is nothing to predict, because the loop over the list goes the same way every time. There is nothing to reorder, because item 500 was never waiting for item 499. What the work wants is the opposite trade: spend the transistors on execution units rather than on the front end, accept that each individual calculation will take longer than a CPU's would, and make it up in volume. That is a GPU, and the rest of this part is that one trade worked out.

The GPU: one instruction, thirty-two results

Given work of that shape, the specialisation is simple to state. The GPU takes the loop from Part 5 and changes one number. Instead of one front end feeding one execution unit, one front end sends each instruction to 32 execution units at once, each with its own registers holding its own numbers. The usual name for one of those 32 units is a lane, and I will use it from here on. One fetch, one decode, 32 executes.

Be precise about what those 32 executes are, because the phrase can be read two ways and only one is right. The execute is not chopped into 32 smaller pieces, and the same calculation is not done 32 times over. It is the same operation done 32 times on 32 different sets of numbers, all in the same tick, by 32 physically separate lanes. The front end issues "multiply register 1 by register 2, add register 3, put the result in register 4" once. Lane 0 does that with the 1, 1 and 0 sitting in its own registers and gets 1. Lane 1 does it with its 8, 6 and 3 and gets 51. Lane 31 does it with whatever it holds. Each lane performs the whole multiply-add from start to finish, exactly as a CPU's execution unit would for a single number, and none of them looks at another lane's registers. What is shared is the fetch and the decode, not the work. Chopping one execute into stages is a different trick, the pipeline from the end of Part 5, and both designs use it. The way a program is written for this is one small program per lane: you write the calculation once, as if for a single number, and the hardware runs that same program on 32 numbers at a time, one per lane, then on the next 32, and so on until the list is done. The demo further down uses exactly those numbers, so you can watch it happen.

The obvious objection is that getting 32 fresh sets of numbers into 32 sets of registers sounds like the real bottleneck, and it is. Keeping the lanes fed is the central problem of GPU design, and the second half of this part and the whole of the next one are about it. But notice first that the load is 32 wide too. "Load register 1 from the address in register 5" is one instruction, issued once, and each lane has its own address in its own register 5. When the program is written the natural way, lane 0 holds the address of item 0, lane 1 the address of item 1, and so on, so the 32 addresses are 32 neighbours, 128 bytes in a row, and the memory system fetches all of them with a single request, much as a CPU fetches a single 64-byte cache line. One request, 32 numbers, one per lane.

How 32 lanes get 32 numbers from one instruction. The front end issues one load. Each lane supplies its own address, and because the program was written so that lane i handles item i, the 32 addresses are 32 neighbours in memory. The memory system fetches the whole 128-byte row with one request and puts one number into each lane's register. If the addresses were scattered it would take up to 32 requests, which Part 10 comes back to.

Three more things keep the objection from being fatal. The memory is built for it: a GPU's memory, later in this part, is designed to hand over wide blocks at a huge rate rather than to answer one small question quickly. The registers are plentiful: the register file is so large that each lane can keep dozens of values at once, so a typical routine does many instructions of arithmetic for every load, and the numbers stay in registers between them. And when a block has not arrived yet, the front end does not wait for it. It switches to another group of 32 lanes whose numbers have arrived, which is the trick Part 7 is built around. When the 32 addresses are scattered instead of adjacent the single request becomes up to 32 of them and the whole scheme slows to a crawl, which is why Part 10 spends a section on how data should be laid out.

The expensive part of the loop, the front end, is paid for once and shared 32 ways, and because fetching and decoding is now so much smaller a share of the work, the GPU can afford to make its front end simple: no branch predictor, no out-of-order machinery, a short pipeline, instructions taken in the order they were written. What it spends the saved transistors on instead is more lanes, and a very large register file so that every lane's numbers are already on the chip when an instruction needs them.

Where the transistors go. Left, one CPU core: most of it is machinery for keeping a single instruction stream moving, and the arithmetic is a small island. Right, one streaming multiprocessor from a GPU: four small front ends, the warp schedulers, each feeding 32 lanes, which are what the spec sheet counts as CUDA cores, plus a tensor core beside each set, a separate unit for matrix arithmetic, and a register file large enough to hold the numbers of thousands of threads at once. Areas are indicative, not measured.

Nvidia's name for one instruction stream running on 32 lanes is a warp, the front end that feeds it is a warp scheduler, and each lane is what the spec sheet counts as a CUDA core. The two words name the same thing: the "16,384 CUDA cores" of an RTX 4090 are 16,384 lanes, and the floor plan labels them both ways. A streaming multiprocessor, the building block of the chip, has four warp schedulers and 128 lanes, and an RTX 4090 has 128 of those blocks, which is where the 16,384 comes from. The tensor core drawn beside each set of lanes is a different unit altogether, despite sharing the word: it multiplies small matrices in one instruction rather than single numbers, it is not counted among the CUDA cores, and Part 11 is about it. A CPU core and a CUDA core are not the same kind of thing at all: one is a complete front end and set of execution units with its own caches, the other is a single lane sharing a front end with 31 others. Part 8 puts a number on how different they are, and Part 9 opens the block up.

What an instruction looks like on each

The clearest way to see the difference is to look at the instructions themselves. Here is the running calculation of this series, y = w × x + b, as a CPU does it one number at a time, in the x86 machine language that Intel and AMD chips speak. The names are ugly but the shape is Part 5's: an operation, and the registers it works on.

vfmadd213ss  xmm0, xmm1, xmm2     ; xmm0 = xmm1 × xmm0 + xmm2, one FP32 number

One instruction, one multiply-add. To do 32 of them the front end fetches and decodes 32 instructions, or, more likely, a loop of a few instructions 32 times. A modern CPU also has wide instructions that treat a register as sixteen numbers side by side, which is the SIMD idea, single instruction, multiple data, and it is where the "one 16-wide unit" in the floor plan comes from:

vfmadd213ps  zmm0, zmm1, zmm2     ; sixteen FP32 numbers at once, in one instruction

Now the GPU. Nvidia's machine language is called SASS, and the equivalent instruction looks like this:

FFMA  R4, R1, R2, R3               ; R4 = R1 × R2 + R3, on every lane of the warp

Read it as a Part 5 instruction and it is nothing special: multiply two registers, add a third, put the result in a fourth. The difference is who executes it. When the warp scheduler issues that FFMA, all 32 lanes of the warp execute it in the same tick, and "R1" means a different physical register in each lane, holding that lane's own w. Thirty-two multiply-adds per instruction, on 32 different sets of numbers, with one fetch and one decode. The programmer never writes "do this 32 times". They write one ordinary-looking program for one thread, and the hardware runs 32 copies of it side by side, in lockstep, which is why Nvidia calls the scheme SIMT, single instruction, multiple threads. Try the three side by side.

The same job on three machines: 32 independent multiply-adds, y = w × x + b, each with its own numbers. Every press of the button fetches, decodes and issues one instruction. Watch how many items each machine finishes per instruction.

Two consequences fall out of that demo, and each gets a part of its own. The first is that the GPU only wins if there are lots of independent items to fill its lanes. Give it one long chain and 31 of every 32 lanes stand idle, which is the race in Part 8. The second is what happens when the 32 copies of the program reach an if and want to go different ways. They cannot, because one front end is issuing one instruction to all of them, and Part 9 shows the trick the hardware uses instead.

Two kinds of memory

The third decision is about memory, and here the two designs part company just as sharply, even though, as Part 5 showed, the cell inside every one of these chips is the same transistor and capacitor.

A CPU's main memory is built for capacity and for being changeable. It comes as sticks, DIMMs, that plug into sockets on the motherboard several centimetres from the processor, and it reaches the processor over a narrow bus: a desktop has two channels of 64 data wires each, a big server perhaps twelve. These channels are not the PCI Express lanes a graphics card plugs into. A memory channel is a set of wires that runs straight from the memory controller on the CPU die to a pair of DIMM sockets and carries nothing but memory traffic. PCI Express is a separate and much narrower link, sixteen lanes to a graphics card, each lane a pair of wires in each direction, that joins the CPU to the GPU rather than either of them to memory. The figure in the section after next puts the three sets of wires side by side. That gives a desktop somewhere around a hundred gigabytes a second and a server a few hundred, with capacities from 32 gigabytes up to a terabyte or more, and you can pull a stick out and put a bigger one in. It is the right shape for a machine that runs one chain at a time and needs a lot of room.

A GPU's memory is built for bytes per second, because 16,384 lanes fed at a hundred gigabytes a second would starve. On a gaming card the DRAM chips, a kind called GDDR, are soldered onto the circuit board in a ring around the die, millimetres away, each with its own wires, and the bus is 384 or 512 wires wide with each wire signalling several times faster than a DIMM's. The RTX 5090 gets about 1.8 terabytes a second that way, from 32 gigabytes that can never be changed. A data-centre GPU goes further with HBM, high bandwidth memory: the DRAM dies are stacked eight or twelve high, wired vertically through the silicon, and the stacks are bonded beside the GPU die on a slab of silicon called an interposer, so that thousands of wires can run a few millimetres between them. An H100 has five such stacks and 3.35 terabytes a second. A Blackwell B200 has eight, for 8 terabytes a second and 192 gigabytes. Same capacitor, same word lines and bit lines, an entirely different building.

Three ways to attach the same DRAM cell to a processor. A CPU's memory sits in sockets centimetres away on a narrow bus, and can be swapped. A gaming GPU has its chips soldered around the die on a bus six to eight times wider. A data-centre GPU stacks the dies and bonds them beside the processor on a slab of silicon, a few millimetres away, with thousands of wires.
Desktop CPU Gaming GPU (RTX 5090) Data-centre GPU (B200)
Memory DDR5 DIMMs GDDR7 HBM3e
Where it sits sockets, centimetres away soldered around the die, millimetres away stacked, bonded beside the die on silicon
Data wires 128 (two channels) 512 8,192 (eight stacks)
Bandwidth about 100 GB/s about 1.8 TB/s about 8 TB/s
Capacity 32 to 128 GB, expandable 32 GB, fixed 192 GB, fixed
Time for one request around 80 to 100 ns around the same around the same

The last row is the one to notice. Eighty times the bandwidth, and the wait for a single number is no shorter. Part 10 goes into the wires and the stacks in detail.

The GDDR generations are worth a table of their own, because every graphics card spec sheet names one and the number after the letters is the easiest thing on the sheet to misread. Each generation raised the speed of every wire, measured in gigabits per second per wire, and the card's bandwidth is that speed times the number of wires. The bus widths have barely changed in a decade.

Generation Speed per wire Example card Bus width Bandwidth
GDDR5 (2008) 7 to 8 Gbps GTX 980 Ti, 2015 384 wires 336 GB/s
GDDR6 (2018) 14 to 16 Gbps RTX 2080 Ti, 2018 352 wires 616 GB/s
GDDR6X (2020, Micron and Nvidia only) 19 to 21 Gbps RTX 4090, 2022 384 wires 1,008 GB/s
GDDR7 (2024) 28 to 32 Gbps shipping, 48 allowed by the standard RTX 5090, 2025 512 wires 1,792 GB/s
GDDR8 no standard yet none

There is no GDDR8 as I write this. The JEDEC standard for GDDR7 was published in March 2024 and runs up to 48 gigabits per second per wire, and the memory makers are still climbing that range: the first cards shipped at 28 and 30, chips rated at 36 and 40 have been announced since, and SK hynix has shown 48. SK hynix's published roadmap lists a "GDDR7-Next", which the industry takes to mean GDDR8, for around 2029 to 2031, with nothing about its speed. So the next few years of gaming cards will be faster GDDR7, not a new letter. GDDR7 also changed how the bits are put on the wire, using three voltage levels instead of two so that every two signals carry three bits, which is part of how it doubled GDDR6's speed without doubling the clock. GDDR6X had done something similar with four levels, and Part 10 comes back to that.

Here is the whole ladder for each machine, size and distance at every rung, with the CPU column for a 2024 desktop core and the GPU column for an H100. The GPU's ticks are longer, since it runs at about 1.8 GHz against the CPU's 5, so both are given in nanoseconds. The next part is about what these numbers mean for how each machine is built.

Level Desktop CPU: size Desktop CPU: time for one access H100: size H100: time for one access
Registers a few kilobytes per core 0.2 ns, one tick 256 KB per SM, 33 MB on the chip one tick
L1 cache 32 to 48 KB per core about 1 ns, 4 to 5 ticks 256 KB per SM, shared with the scratchpad about 20 ns, 30 to 40 ticks
L2 cache 1 to 2 MB per core about 3 ns, 14 ticks 50 MB, shared by the whole chip about 140 ns, 260 ticks
L3 cache 32 to 96 MB, shared by all cores about 10 ns, 40 to 50 ticks none
Main memory 32 to 128 GB of DDR5 80 to 100 ns 80 GB of HBM3 about 250 to 300 ns, nearly 500 ticks

Read the GPU column and something looks wrong. Its L2 is slower than the CPU's main memory, its L1 is slower than the CPU's L3, and a trip to HBM takes three times as long as a trip to a DIMM. That is not a mistake. GPU memory is not built to answer one request quickly, and the next part explains why the GPU can afford that and a CPU cannot.

One memory for both

There is a third arrangement, and it has become common enough to deserve its own paragraph: give the CPU and the GPU one shared pool of memory instead of one each. Apple's M-series chips work this way, with the CPU cores, the GPU cores and a ring of ordinary LPDDR5 chips all on one package sharing one address space, which is what "unified memory" on a Mac means. Nvidia's DGX Spark, a desktop box built around its GB10 chip, does the same with 128 GB of LPDDR5X at 273 GB/s, and AMD's Ryzen AI Max laptop chips are a third example. A Mac mini with an M4 Pro has 273 GB/s and up to 64 GB, a Mac Studio with an M3 Ultra has 819 GB/s and up to 512 GB. Here are the two arrangements side by side, with every set of wires labelled.

Who is wired to what. Left, a conventional PC: the CPU reaches its DIMMs over two memory channels of 64 data wires each, the GPU reaches its own GDDR chips over 512 wires, and the two processors are joined by a PCI Express link of 16 lanes, which is neither of those and far narrower. A number the GPU needs has to be copied from the DIMMs across that link into the GDDR before the GPU can touch it. Right, unified memory: CPU cores and GPU cores on one package share one memory controller and one pool of chips, so nothing is copied and there is no link between them to cross. Band widths hint at wire counts and are not to scale.

The advantages are real. There is no copying. On a system with a discrete GPU, every number the GPU works on has to be sent across the PCI Express link from the CPU's memory into the GPU's, and back again afterwards, at a few tens of gigabytes a second, and Part 12 will show how much of a program's life goes on that. With one pool, the GPU simply reads what the CPU wrote. And the GPU gets to use all of the memory, which is why a Mac Studio can hold a model of several hundred billion parameters in one box when no single graphics card can. A 512 GB pool costs a fraction of what 512 GB of HBM would, draws a fraction of the power, and fits under a monitor.

The disadvantage is the row that keeps coming up. That pool is DIMM-class memory, a few hundred gigabytes a second, between a third and a thirtieth of what GDDR or HBM delivers, and both processors are drawing on it at once. Running a large language model is, as Part 10 will show, almost entirely a matter of how fast the weights can be streamed from memory, so a model that fits in a Mac Studio's 512 GB is read at 819 GB/s, while a model spread across GPUs with HBM is read at 8 TB/s each. The unified machine wins on what fits and on price, and loses by an order of magnitude on how fast it runs once it does fit. The memory is also fixed at purchase, and the GPU on these chips is a small one, a fraction of a data-centre part. Nvidia's own answer for the data centre is a hybrid: the Grace Hopper and Grace Blackwell superchips keep a CPU with a few hundred gigabytes of LPDDR5X and a GPU with HBM, joined by a 900 GB/s link into one address space, so that the GPU can reach the CPU's larger, slower pool when its own fast one is full. Part 13 comes back to that.

What the number on the box means

A stick of desktop memory is sold as "DDR4-3200" or "DDR5-6400", and shops call it 3200 MHz or 6400 MHz RAM. It is worth knowing what that number is and is not, because it answers a question people reasonably ask: does faster RAM mean the processor waits less?

The number is the count of transfers per second on each data wire, in millions. DDR stands for double data rate, meaning the memory sends one value on the rising edge of its clock and another on the falling edge, two per tick, so DDR4-3200 actually runs a 1,600 MHz clock and DDR5-6400 a 3,200 MHz one. The "MHz" on the box is a habit rather than a clock speed. Multiply the transfer rate by the width of the bus and you get the bandwidth: 3,200 million transfers times 64 wires is 204,800 million bits a second, or 25.6 gigabytes a second per channel, and 51.2 for DDR5-6400. Two channels doubles it again. So the number on the box is a bandwidth number. Double it and twice as many bytes arrive per second once they are flowing.

What it does not do is shorten the wait. The other number on the box, the CAS latency or CL, is how many clock cycles pass between the memory being asked for a column of an open row and the first data coming out. It is quoted in cycles, and it has risen with every generation, from CL16 on DDR4-3200 to CL32 or more on DDR5-6400, because the clock doubled and the physics of the capacitor did not change. Sixteen cycles at 1,600 MHz is 10 nanoseconds. Thirty-two at 3,200 MHz is also 10 nanoseconds. Add the time to open the row, the trip from the core through the caches to the memory controller and back, and the total comes to somewhere between 80 and 100 nanoseconds for either stick, and has for twenty years. This is the fact the next part is built around, met a little early: you can buy as much bandwidth as you like, and the latency stays where it is.

What the speed on the box buys you. Pick a memory, or move the sliders. Bandwidth is transfers per second times the width of the bus. The wait for the first byte is the CAS latency in clock cycles, turned into nanoseconds by the clock.

So, does faster RAM make the processor wait less? For a single number, hardly at all. For a stream of data, yes, in proportion, because the wait is paid once and then the bytes arrive twice as fast. Which of those two describes your program is the whole question, and it is the question the next two parts are about. A CPU running a chain of dependent decisions is usually waiting for single numbers, which is why a faster stick makes so little difference to it. A GPU multiplying matrices is drinking from a stream, which is why its memory is built the way it is.

Where this leaves us

One fact about the work, and three decisions that follow from it. The fact: the same short routine, applied to millions of items that do not depend on each other. The decisions: one front end feeding 32 lanes instead of one execution unit, which is where the width comes from. A simple loop with nothing around it, which is where the transistors for all those lanes come from. And memory built for bytes per second rather than for size or for a quick single reply, which is what keeps the lanes fed. Everything else that makes a GPU a GPU is a consequence, and the first consequence to face is the one this part kept running into: whatever the bandwidth, a trip to memory takes hundreds of ticks, and a processor that adds in four has to do something about that. What a CPU does about it, what a GPU does about it, and why they choose opposite answers, is the next part.

Next: The Memory Wall: why a processor that can add in a third of a nanosecond spends most of its life waiting for memory, the ladder of caches that tries to hide it, and what each kind of processor does while it waits.