The news. On August 25, 2026, Cerebras published a Hot Chips 2026 deep dive describing CS-5, targeted for 2027, and the planned CS-6. CS-5 is designed for up to 10,000 output tokens per second per user on Gemma 4 31B and gpt-oss-120b, and up to 5,000 tok/s per user on multi-trillion-parameter models such as Kimi and GPT-5.6 Sol, at a target of 3 million tok/s per megawatt. The described mechanism is locality: a Nexus rack holds three wafer-scale compute backpacks, high-volume tensor and expert traffic stays inside each wafer, and multi-system execution pipelines the lower-volume activations between wafers. The current WSE-3T is stated at 53.5 PB/s of aggregate on-wafer fabric bandwidth, and CS-6 adds tightly integrated 3D-stacked DRAM to expand capacity while preserving that locality. Read the deep dive →
Picture a copyist at a desk, writing out a manuscript line by line. There is a catch: they are not writing from memory. Every single line requires consulting the master copy — the whole of it, not a bookmark — before a word of it can be set down. Now put the master copy in a vault down the hall. The copyist can write quickly, they are trained and their pen is fast, but between one line and the next they walk to the vault, find the passage, and walk back. Watch them for an hour and the pen is barely the story: the walk is what sets the pace, and buying them a faster pen changes almost nothing.
That desk is a decoding accelerator, and the vault is where your model lives. The weights are the master copy, and generating one output token means reading essentially all of them once — an enormous quantity of data for a very small amount of arithmetic. That ratio is exactly what the roofline model is built to expose, and it places decoding firmly on the memory side of it. A decode step is memory-bound, so a single user's token rate is set by how fast weights reach the math units, not by how fast the math runs — which is why per-user speed does not improve nearly as much as raw FLOP counts suggest it should. On a GPU those weights sit in HBM: physically next to the die, connected across a package boundary, and re-read step after step after step. The walk is short in human terms and enormous in silicon terms, and it is paid on every token.
Cerebras' bet is to delete the walk rather than shorten it: build the processor out of an entire wafer, so that what a decode step reads and the math units reading it sit on one piece of silicon. The deep dive states the rule for the traffic that matters most — high-volume tensor and expert communication stays inside a wafer — and describes the planned CS-6 as combining wafer-scale SRAM and compute with 3D-stacked DRAM. In the metaphor, the master copy is no longer in a vault at all — it lies open on the desk, and the copyist reads from it without standing up. The figure the deep dive gives for this is 53.5 PB/s of aggregate on-wafer fabric bandwidth on the current WSE-3T — a figure for today's silicon, not a CS-5 specification: not a claim that the arithmetic got faster, but a claim about how much data can be in motion inside the wafer at once. The design premise is that if you never leave the silicon, the trip that dominates decoding stops existing.
The bill for that locality is capacity. On-die SRAM is fast precisely because it is close and small, so a wafer holds far less than a rack of DRAM does, and capacity is the limit a single wafer runs into first. This is where the second half of the mechanism matters. The described Nexus rack holds three modular wafer-scale compute backpacks, and the rule for what may cross between them is a rule about volume: high-volume tensor and expert communication stays inside a single wafer, while what gets pipelined between wafers is the lower-volume activations. Back at the scriptorium, this is several desks in one room, each with its own portion of the manuscript lying open, passing a short note — "here is the sentence so far" — from one to the next. The pages never travel. The big thing stays put and the small thing travels — the same locality rule applied one level up, from inside a wafer to between them.
CS-6's stated addition follows the same logic one step further. Rather than solving the capacity limit by going back out to distant memory, the plan described is tightly integrated 3D-stacked DRAM: more storage bonded directly to the compute, expanding capacity while preserving locality. The cabinet is wheeled up beside the desk instead of the copyist being sent back down the hall. Whether the resulting bandwidth behaves like the SRAM it supplements is not something the post quantifies, and CS-5 is targeted at 2027 rather than shipping — so this is an announced design direction, not a measured result.
Hold one number fixed and the shape of the tradeoff becomes arithmetic rather than rhetoric. Take a 31B-parameter model at two bytes per parameter: about 62 GB of weights, all of which a memory-bound decode step must read to produce a single token. Price that same 62 GB two ways. Against a single modern accelerator's off-chip HBM at roughly 8 TB/s, one token's worth of weight-reading takes about 7.8 ms, capping that accelerator near 130 tokens per second for that user. Against the 53.5 PB/s aggregate on-wafer figure, the same 62 GB clears in roughly 1.2 microseconds — about 6,700× less time for the identical work. (Illustrative: the 8 TB/s HBM figure is a textbook order-of-magnitude, not a number from the Cerebras post, and both sides ignore batching, sharding and overlap, so treat these as a ratio between two memory systems rather than as predicted throughput.) Neither side did less arithmetic; the entire difference is how far the weights had to travel — which is why the stated CS-5 target is a per-user figure of 10,000 tok/s, the metric that a locality change moves and a FLOP change largely does not.
| Where the weights live | GPU + off-chip HBM | Wafer-scale (Cerebras) | CS-6 (planned) |
|---|---|---|---|
| Distance from the math units | across a package boundary | on the same wafer | 3D-stacked, bonded to the compute |
| Stated bandwidth figure | ~8 TB/s (illustrative, setup-dependent — not from the source) | 53.5 PB/s aggregate on-wafer fabric (WSE-3T) | not quantified in the post |
| What limits model size | HBM capacity per accelerator | on-wafer SRAM capacity | stacked DRAM capacity, locality preserved |
| What crosses the slow link | weights, every step | activations are pipelined between wafers; tensor and expert traffic stays on-wafer | |
| Availability | shipping today | CS-5 targeted for 2027; CS-6 planned | |
What makes this worth reading past the product names is that it is a clean instance of a decision every serving stack faces, at a scale most of them cannot reach. The usual levers work on the same bottleneck from the other end: quantization shrinks the manuscript so the walk carries fewer pages, batching amortizes one walk across many readers, and caching avoids repeating a walk already made. Wafer-scale attacks the distance itself — and reading it that way tells you what to check when the 2027 numbers arrive: not the FLOP count, but what still has to cross a boundary, and how much of it there is.
Goes deeper in: GPU & CUDA → Memory Hierarchy → HBM: Where Your Model Lives
Related explainers
- Vera Rubin NVL72 — The NVLink rack domain — the rival answer to the same problem: widen the fast domain across a rack instead of collapsing it onto one wafer
- NVIDIA AI Factories — Tokens-per-megawatt as a serving metric — the metric CS-5's 3 million tok/s per megawatt target is stated in
- MobileMoE — DRAM-aware MoE scaling — the identical locality tradeoff at the opposite end of the scale, on a phone