Instinct MI300

On 6 December 2023, a year after the H100 of Part 24 began shipping, AMD launched the Instinct MI300X. Part 10 has already taken its package apart: eight compute dies stacked on four base dies, 304 compute units, 256 MB of Infinity Cache, eight stacks of HBM3, 153 billion transistors. This part is about what the chip was for and what happened when people used it. The short version is one number. The H100 had 80 GB of memory. The MI300X had 192.

AMD's data-centre GPUs had existed for six years by then, and almost nobody had trained a language model on one. Within a year of the MI300X's launch, Meta was serving its largest model on them, Microsoft was running parts of its Copilot services on them, and AMD had sold more than $5 billion worth. It is worth seeing how a company that had been a footnote in AI got there, and why it did not get further.

Before MI300: the supercomputer chips

The MI300 grew out of a line built for science rather than AI. The MI100 of November 2020, the first CDNA chip, had 120 compute units, 32 GB of HBM2 and AMD's first matrix cores, and 11.5 TFLOPS of 64-bit arithmetic on its ordinary lanes, more than Nvidia's A100 of the same year managed on its own. The MI250X of November 2021 put two dies in one package, each a complete GPU with its own 64 GB of HBM2e, joined by 400 GB/s of Infinity Fabric links between them. To software it was two GPUs, not one, a detail that matters later. Together the pair had 220 compute units, 128 GB at 3.2 TB/s, and 47.9 TFLOPS of 64-bit vector arithmetic, 95.7 on the matrix cores, at 500 to 560 W.

Sixty-four-bit arithmetic is what supercomputers buy, and on that measure the MI250X was ahead of anything Nvidia sold until the H100 a year later, and in raw FP64 throughput ahead of the H100 too. It went into Frontier at Oak Ridge: 9,408 nodes, each one AMD EPYC processor and four MI250X, 37,632 GPUs in all. In May 2022 Frontier ran the standard benchmark at 1.102 exaflops, the first computer past a billion billion 64-bit operations a second, drawing 21.1 MW. Upgrades since have taken it to 1.353 exaflops, and on the June 2026 list it stands third.

The supercomputer contracts did something besides make headlines: they paid for the engineering the MI300 needed. The next machine, El Capitan at Lawrence Livermore, for which AMD was chosen in 2020, ended up built on a new kind of chip, one that put CPU cores and GPU compute dies on the same package around a single memory. That chip became the MI300A, and the MI300X is its sibling.

One package, two chips

The MI300 package is a kit. Four I/O dies form the base, each with room for two compute dies on top, and AMD built two products from the same parts. The MI300X puts eight GPU compute dies (XCDs) on the four bases. The MI300A puts six XCDs on three of them, and on the fourth it puts three CPU dies taken from AMD's EPYC server processors, 24 Zen 4 cores in all. Both have eight stacks of HBM3 at 5.3 TB/s; the MI300A's stacks are smaller, 128 GB in total.

The MI300A is the stranger and in some ways the more interesting of the two. Part 14 described the CPU and GPU as two machines with two memories, joined by a slow road, so that every program begins by copying its data from one to the other. In the MI300A there is one memory. The CPU cores and the GPU compute units read and write the same 128 GB of HBM, with the same addresses, so a scientific code that alternates between serial work on the CPU and parallel work on the GPU stops paying for the copies in between, and the CPU gets HBM bandwidth that no ordinary server processor has. It is the same idea as the phone and laptop chips of Part 32, at 550 watts.

El Capitan was built from it: at its debut, 11,136 nodes of four MI300A each, 44,544 chips. In November 2024 it took first place on the supercomputer list at 1.742 exaflops, and a year later reached 1.809 exaflops while drawing 29.7 MW, about 61 billion 64-bit operations per joule. On the list published in June 2026 it was overtaken by LineShine, a new Chinese machine at the national supercomputing centre in Shenzhen built entirely from CPUs, which reached 2.198 exaflops on 42 MW. El Capitan remains first on the list's mixed-precision benchmark, at 16.7 exaflops, which is the closer measure of how these machines are now used for AI.

The memory argument

Back to the MI300X, and to the number. Part 12 explained why memory decides almost everything about running a language model: every generated token reads every weight, so bandwidth sets the speed, and the model has to fit before it can run at all. A model's weights take two bytes per parameter at FP16, one at FP8. A 70-billion-parameter model at FP16 is 140 GB. That does not fit on one H100. It fits on one MI300X, with room left over for the cache of past tokens that every conversation builds up. Meta's Llama 3.1 405B at FP16 is about 810 GB, which needs more than one eight-GPU H100 server, 640 GB, but fits in one eight-GPU MI300X server, 1.5 TB.

Fewer GPUs per model means fewer bytes crossing the links between them for every token, which is the communication cost of Part 15, and fewer servers to buy. Try some models below.

Try Llama 3.1 405B at FP16 and the H100 needs thirteen GPUs, which means two servers, while the MI300X needs six, inside one. Switch to FP8 and both fit in one, which is part of why FP8 mattered so much to Nvidia's customers. The comparison is not frozen, either. Nvidia's H200 of 2024 raised the same chip's memory to 141 GB, and every generation since has raised it again on both sides, but at its launch the MI300X had more than twice the memory of anything Nvidia shipped.

That advantage found customers. Microsoft made MI300X virtual machines generally available on Azure in May 2024 and said it was running GPT-4-based services on them, including the Azure OpenAI Service and some Copilot features. On AMD's results call in February 2025, Lisa Su said that Meta had served its 405-billion-parameter Llama model on meta.ai exclusively on MI300X. And in AMD's first entry in the MLPerf inference benchmark, in August 2024, a single MI300X held and ran the whole of Llama 2 70B, while eight of them came within a few per cent of Nvidia's eight-GPU H100 system on the same test. The newer H200 was well ahead of both. AMD's Instinct sales passed $5 billion in 2024.

What held it back

On paper the MI300X had more of almost everything than the H100: 1,307 against 989 TFLOPS of dense FP16 matrix arithmetic, 5.3 against 3.35 TB/s, 192 against 80 GB. In practice it did not win the way those ratios suggest, for two reasons.

The first was software, the moat of Part 14 seen from outside it. In December 2024 the research firm SemiAnalysis published months of benchmarking and concluded that the MI300X's performance was held back by AMD's software. On the public releases of AMD's software stack, a large matrix multiply in BF16 reached about 620 TFLOPS of the 1,307 on the spec sheet, while an H100 reached about 720 of its 989, and the H100 and H200 won every training test they ran. AMD's engineers spent the next two years closing that gap. ROCm 7, released in September 2025, came with AMD's claim of three and a half times the inference speed of ROCm 6 on the same eight MI300X, and in January 2026 the vLLM project, one of the most widely used open-source serving engines, made ROCm a first-class platform alongside CUDA. Each of those is a sign of how far behind the starting point was.

The second was how the eight GPUs in a server talk to each other. An MI300X has seven Infinity Fabric links of 128 GB/s, 896 GB/s in all, almost exactly an H100's 900 GB/s of NVLink. But the H100s in a server are joined through NVSwitch chips, so any one GPU can send its full bandwidth to any other. The MI300X server has no switch. Each GPU is wired directly to each of the other seven, one link apiece, so any single pair can exchange at most 128 GB/s, 64 in each direction, a seventh of the total. For the all-to-all traffic of splitting one model across eight GPUs that is fine. For a pair of GPUs that need to talk to each other a lot, it is a seventh of what the H100 offers. Part 15 draws the two arrangements side by side. And in 2025, when Nvidia's GB200 NVL72 rack joined 72 GPUs into one NVLink domain, AMD had nothing comparable until the Helios rack of Part 30.

One GPU, or eight

There is a third difference that is not a weakness so much as a choice. An MI300X is eight compute dies, and AMD lets the owner decide how many GPUs it is. By default all eight XCDs act as one GPU with one pool of 192 GB, which AMD calls SPX mode. In CPX mode each XCD becomes a GPU of its own, so an eight-GPU server reports sixty-four, and the memory can be split into four separate pools as well. Small models then get a whole slice of GPU each instead of sharing one, which is Nvidia's MIG partitioning from Part 16 done along the chiplet boundaries. The Wikimedia Foundation, which runs Wikipedia, publishes how it uses this on its own MI300X servers.

The MI325X

In October 2024 AMD followed with the MI325X: the same package, the same 304 compute units, and the same peak arithmetic, with the HBM3 replaced by 256 GB of HBM3E at 6 TB/s, and the power raised from 750 to 1,000 W. AMD had shown it at 288 GB earlier in the year and settled on 256. It shipped from the end of 2024. It is the clearest example in this series of a generation that changed nothing but the memory, and it was worth doing, because for inference the memory is most of the chip. Its photograph is in Part 10.

Where this leaves us

The MI300 is the chip where AMD's long habits paid off. The chiplets and stacking it had been building since the Fiji interposer let it put more silicon and more memory in one package than a single-die design could, and the supercomputer contracts paid for the engineering. One kit made both a GPU and a CPU-and-GPU with one memory, and the second of those runs one of the two fastest computers on the 2026 list. The first found its way into AI through a single advantage: 192 GB when the competition had 80, enough to hold models whole that had to be split elsewhere.

What it did not have was Nvidia's software, Nvidia's switch, or Nvidia's rack, and the next two AMD parts are about closing those gaps. First, Intel's answers to Hopper, which tried two quite different ideas, and then Nvidia's next step, two dies in one package.

Next: Intel in the Data Centre: Ponte Vecchio's forty-seven tiles and the Aurora exascale machine built from them, Gaudi, an AI accelerator that was not a GPU, and Crescent Island, a 2026 card that trades HBM for 160 GB of laptop memory.