What is Model Runner V2 in vLLM 0.25?

Model Runner V2 (MRv2) is vLLM's rewritten model executor — the component that runs the forward pass on each step. In 0.25 it becomes the default path for dense models and unifies quantization, dynamic speculative decoding, multimodal prefixes, and KV transfer into one code path whose launch pattern is static enough to capture as a full CUDA graph.

Why did vLLM remove PagedAttention?

The release notes state the removal and that vLLM 0.25 now supports full CUDA graphs, but don't detail the internal rewrite. The most plausible reading: modern attention backends (FlashAttention, FlashInfer) now do the block-table paging natively inside the kernel, which makes the standalone PagedAttention layer redundant and was likely part of what had kept vLLM from capturing the whole decode step as a replayable CUDA graph. So the removal pairs with Model Runner V2 and the move to full CUDA graphs — treat the causal link as the likely mechanism, not a claim spelled out in the notes.

Is paged KV cache gone now?

No — only the standalone PagedAttention kernel is gone. The idea of paged KV (logical blocks mapped to scattered physical memory) moved down into the attention backends that most serving stacks already use. The concept won so thoroughly it became a built-in feature rather than a separate layer, which is why the standalone kernel could be retired.

LLM· 2026-07-12

vLLM 0.25 makes Model Runner V2 the dense default — Retiring PagedAttention — What does it mean?

TL;DR

What is it: The vLLM 0.25 release makes Model Runner V2 (MRv2) the default path for dense models and, in the same breath, removes PagedAttention — the KV-cache paging kernel that made vLLM famous in 2023.
Why it’s needed: The release's headline is full CUDA graphs — recording the GPU's whole per-step routine once and replaying it, instead of the CPU dispatching many tiny kernels live — and, by the most plausible reading of the notes, retiring the bespoke PagedAttention layer is what clears the way.
vs previous: The old runner could only capture CUDA graphs for a padded ladder of fixed batch sizes; vLLM 0.25 reports full CUDA graphs for the whole path, dynamic speculative decoding included — a capability the notes state plainly, even though they don't detail the internal rewrite.

Jargon

PagedAttention: vLLM's founding trick: store the KV cache in fixed-size blocks scattered across GPU memory and keep a per-sequence block table mapping logical positions to physical blocks — virtual memory for the KV cache. It killed memory fragmentation. A custom attention kernel read the block table to gather the scattered blocks.
Model Runner V2 (MRv2): vLLM's rewritten model executor — the component that actually runs the forward pass each step. In 0.25 it becomes the default for dense models and folds quantization, speculative decoding, multimodal prefixes, and KV transfer into one path.
CUDA graph: A recording of a whole sequence of GPU operations that can be replayed in a single launch. It removes the per-kernel launch overhead the CPU otherwise pays for every op — the dominant tax during decode.
Kernel launch overhead: The fixed CPU cost of telling the GPU to run one kernel. Decode fires many tiny kernels per token, so this overhead, multiplied out, can rival the actual math.
FP8 / INT4 / NVFP4: Low-precision number formats vLLM 0.25 adds KV-cache and weight paths for. Fewer bits per value means more tokens fit in the same memory, at some accuracy cost — see how numbers shrink.
Speculative decoding: Guess several tokens with a cheap draft model, then verify them in one pass of the big model. MRv2 makes its dynamic variant compatible with full CUDA graphs, which older runners could not.
NCCL symmetric memory: An NVIDIA collective-communication feature for multi-GPU serving where each GPU exposes a symmetric memory region, cutting synchronization cost when tensors or KV blocks move between devices.

The news. On July 11, 2026, vLLM 0.25.0 shipped with Model Runner V2 as the default path for dense models and one blunt line in the release notes: "PagedAttention has been removed." The same release folds in dynamic speculative decoding compatible with full CUDA graphs, a unified Streaming Parser Engine, FP8/INT4/NVFP4 quantization paths, KV offloading, and distributed-serving fixes around NCCL and prefill/decode KV transfer. Read the release →

Picture an old telephone exchange. Every call has to pass through a switchboard operator who reads a plugboard, finds the two scattered lines the call needs, and physically patches them together — then waits for the next request and does it again. That operator is PagedAttention. When vLLM launched in 2023, the KV cache was scattered across GPU memory in fixed-size blocks, and PagedAttention was the operator who, every single decode step, read a block table and gathered the right blocks. It was a brilliant hire: it ended the memory fragmentation that used to force each request to reserve one giant contiguous slab of GPU memory up front.

But an operator is a person in the loop, and a person in the loop is slow. Because the operator patches a different set of lines on every call, the exchange can never settle into a fixed routine — and a fixed routine is exactly what the next optimization needs. That optimization is the CUDA graph: instead of the CPU calling out each GPU operation live, you record the whole per-step routine once and replay it in a single launch. Recording only works if the routine is the same every time. In practice, older vLLM could only capture graphs for a padded ladder of fixed batch sizes and ran everything else the slow, live-dispatched way.

PagedAttention borrowed its whole design from how an operating system fakes unlimited RAM — virtual pages mapped to scattered physical frames through a page table. The diagram below is that idea, and it is the machinery vLLM is now retiring:

OS Virtual Memory — 4 programs sharing physical RAM

What each program sees (virtual)

Chrome

VS Code

Slack

Spotify

Each app sees pages 0, 1, 2 — contiguous

Physical RAM (actual)

Pages scattered — C0 at slot 2, C1 at slot 7...

Page table translates:Chrome v0 → p2, VS Code v0 → p0, Slack v0 → p3

vLLM 0.25's move is to fire the switchboard — not because paging was wrong, but because the phone network learned to route itself. The release notes state the outcome plainly — "PagedAttention has been removed," Model Runner V2 as the dense default, and dynamic speculative decoding now compatible with full CUDA graphs — without spelling out the internal rewrite. The most plausible reading of why those go together: modern attention backends (FlashAttention, FlashInfer) now read block tables and gather paged KV inside the kernel, so the bespoke PagedAttention layer became a redundant middleman, and MRv2 could then unify the forward pass into one path static enough to capture the whole decode step as a replayable CUDA graph. Treat that causal chain as the likely mechanism behind the sourced facts, not a line-by-line claim from the notes.

Walk the launch-overhead math on one decode step (illustrative — the release reports the capability, not timings). Say a single decode step fires 60 tiny GPU kernels, each carrying ~5 µs of CPU launch overhead before any math runs: that is ~300 µs of pure dispatch per token. At those numbers, generating 200 tokens spends ~60 ms just launching kernels — often more than the arithmetic itself. Capture that step as one CUDA graph and all 60 kernels replay from a single launch: dispatch collapses from ~300 µs toward ~5 µs per token, and the ~60 ms of pure overhead nearly vanishes. That collapse is the whole reason "full CUDA graphs" is the headline of the release.

Era	Who does KV paging	CUDA-graph coverage	Cost
PagedAttention (2023–2025)	a bespoke vLLM gather kernel	partial — a padded ladder of fixed batch sizes	live kernel dispatch on the un-captured paths
Model Runner V2 (vLLM 0.25) [release]	the attention backend, natively	full — the whole decode step, spec decoding included	one launch per step (~5 µs, illustrative)

The honest caveat: "PagedAttention has been removed" is a statement about the kernel, not about the idea. Paged KV — logical blocks mapped to scattered physical memory — is more alive than ever; it simply moved down a layer, into the attention backends most serving stacks already depend on. That is the quiet lesson of this release: PagedAttention didn't lose, it won so completely that its trick became a built-in feature and the standalone layer stopped earning its keep — the same way the switchboard operator's job didn't disappear, it got wired into the exchange. When a technique is good enough, the reward is to be absorbed and forgotten.

Goes deeper in: LLM Internals → Paged Attention → PagedAttention in Practice

Related explainers

vLLM v0.20 — FlashAttention 4 packing — the previous vLLM release's headline kernel; MRv2 is the runner that now drives kernels like FA4 under one capturable path.
vLLM v0.20 — TurboQuant 2-bit KV cache — KV-cache compression; vLLM 0.25 broadens the same idea with FP8/INT4/NVFP4 KV paths.
AMD ATOM + ATOMesh — Prefill/decode disaggregation — another serving-stack rework; 0.25 also touches the prefill/decode KV transfer MRv2 unifies.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based