Hopper

In 2022 Nvidia shipped two flagship chips on the same TSMC 4N process with nearly the same number of transistors. One was the AD102 of Part 13, 76.3 billion transistors for gamers. The other was the GH100, 80 billion transistors for data centres, sold as the H100. If you want to know what "an AI chip" means as opposed to "a graphics chip", there is no better way than to put those two side by side, because almost every transistor was spent differently.

Hopper is also, as I write in 2026, the chip that trained most of the models you have used. Meta's Llama 3 was trained on around sixteen thousand H100s. The same is true, as far as anyone outside the labs can tell, of every frontier model of 2023 and 2024. Blackwell has since taken over the biggest installations, but the H100 is the reference point everything else is measured against, and its whitepaper reads like a checklist of the problems raised in Parts 7 through 11.

Two chips, one process

Read down the table and Hopper's priorities are plain. The lane count is almost identical, which tells you the lanes are not where the generation's effort went. The memory is the first big divergence: 80 GB of HBM3 in five stacks on the package, delivering three times the bandwidth of the gaming card, exactly the move Part 8 said a data centre chip would make. Because the bandwidth is there, the L2 can be smaller than Ada's, since it has less to make up for. (The GH100 die carries 60 MB and the H100 product enables 50, the same yield story as the SMs.) Then NVLink at 900 GB/s, which Ada does not have at all, for the multi-GPU work of Part 11. Then the deletions: no ray tracing, no display, no video encoders. And then two things that only a data centre customer asks for, fast 64-bit arithmetic for scientific computing, at half the FP32 rate where the gaming chip manages a sixty-fourth, and tensor cores that do roughly four times the matrix work of Ada's per SM per tick.

The 814 mm² die sits right against the reticle limit from Part 2. Hopper is as big as a single chip can be, and the extra area over Ada went to the HBM interfaces around the edges, the wider tensor cores, the FP64 units, and NVLink. There was nowhere left to grow, which is the problem Blackwell inherits.

The Transformer Engine

Hopper's fourth-generation tensor cores support FP8, as Ada's do, but Hopper wraps them in something Nvidia calls the Transformer Engine, and it is worth separating the marketing from the mechanism. The mechanism is the scale-factor problem from Part 9. An FP8 number has three mantissa bits and a range that tops out at 448, so using it safely means keeping every tensor scaled into that range, and the right scale changes from layer to layer and from step to step as training proceeds. The Transformer Engine is hardware that records the statistics of each tensor as it flows through, plus software that uses those statistics to choose the scale for the next step and to decide, per layer, whether FP8 is safe or whether to fall back to 16 bits. It is what makes FP8 training practical rather than a research curiosity, and the result is the 1,979 TFLOPS in the table, double the FP16 figure for the same silicon.

SMs that cooperate

The programming model of Part 10 has a hard wall in it: threads in a block can share memory and synchronise, and threads in different blocks cannot, because the blocks may be on different SMs. That wall dates from 2006, and for matrix multiplies it hurts, because the ideal tile for a modern tensor core is bigger than one SM's shared memory can hold.

Hopper adds a level to the hierarchy. A thread block cluster is a group of blocks, up to eight as standard or sixteen on the H100 if the program opts in, that the scheduler guarantees to place on SMs within the same GPC at the same time. Blocks in a cluster can read and write each other's shared memory over a new network inside the GPC, a feature called distributed shared memory, and they can synchronise with each other. In effect, the SMs of a GPC can pool their shared memory into a tile several times larger than any one of them could hold. Thread, warp, block, cluster, grid. It is the first change to the shape of the programming model since CUDA began.

The second addition fixes a cost that Part 8 skated over. Loading a tile into shared memory is not free for the lanes: each thread has to compute an address, issue a load, wait, and store, which is dozens of instructions per thread that do no arithmetic. Hopper adds the Tensor Memory Accelerator (TMA), a unit that takes a single description of a multi-dimensional tile, "this 128 by 128 block of that matrix", and copies it between HBM and shared memory on its own, while the lanes get on with the previous tile. Paired with new asynchronous barriers that let threads wait for the copy to complete only when they need it, this is what lets a Hopper matrix multiply keep its tensor cores fed close to their peak, and it is the reason the fastest attention kernels of the last few years are Hopper-specific.

The rest of the checklist

Hopper's whitepaper has a few more items that matter in data centres and nowhere else, and they belong in the picture. MIG, from Ampere, still slices one H100 into up to seven isolated GPUs for sharing. Hopper adds confidential computing, the first on a GPU, so that a model and its data can run encrypted in memory on a cloud machine that the cloud operator cannot inspect. A set of DPX instructions speeds up dynamic-programming algorithms, the kind used in genomics and route planning, by up to seven times. And the memory story continued after launch. The H200 in 2024 was the same GH100 die with 141 GB of HBM3e at 4.8 TB/s, and on inference workloads, which are memory-bound as Part 8 explained, it was substantially faster than the H100 with not one transistor of the compute changed. The GH200 superchip paired the same GPU with a Grace CPU over NVLink-C2C, the arrangement from Part 11.

Where this leaves us

Hopper is what you get when you take the SM of Part 7 and spend the entire rest of the transistor budget on feeding it: HBM3 for bandwidth, a Transformer Engine for fewer bits per number, thread block clusters and the TMA for bigger tiles and fewer wasted instructions, and NVLink for the numbers that live on other chips. Every feature is aimed at the roofline of Part 8, and nearly none of it would help a game. Beside Ada it makes the definition of an AI chip concrete: the same lanes, the same tensor core generation, and everything around them rebuilt for matrix arithmetic on numbers that do not fit on the die.

It also marks the end of something. At 814 mm², Hopper is as large as a single die can be made. Every gain from here has to come from somewhere other than a bigger chip, and Nvidia's answer, two years later, was to stop thinking of the chip as the unit at all.

Next: Blackwell: two dies on one package, 208 billion transistors, FP4, the 72-GPU rack, and the RTX 50 series that brought the same generation to gamers.