vLLM 0.30 ships Fast Start — Persistent GPU weight cache via CUDA IPC — What does it mean?

The news. On September 22, 2026, the vLLM project published v0.30.0, a release of 762 commits from 315 contributors. One highlight is Fast Start: a persistent per-GPU weight-cache daemon holds post-quantized, TP-sharded weights in GPU memory, and restarting engines map them over CUDA IPC with --load-format ipc_cache instead of reloading from disk. In the same release it gained support for FP4 checkpoints and multi-node tensor parallelism. Read the release notes →

Picture a theatre where the whole set is taken down after every show. Before the next performance the crew drives to the warehouse, carries every flat back, paints it, and cuts it to fit the stage — and the audience waits the whole time. Fast Start keeps one stagehand on each stage overnight, standing guard over a finished set, so the next cast only needs the key to the stage door.

In serving terms, the set is the weights. Every forward pass reads the model's weights from GPU memory, so nothing can run until they are there. A cold start does three jobs in order: read the checkpoint files from storage, convert them into the kernel's quantized layout, and split each matrix into one slice per GPU for the tensor-parallel workers the executor drives. A restart normally repeats all three, even when the weights have not changed at all.

With Fast Start, a separate weight-cache daemon does those three jobs once per GPU and then keeps the result in that GPU's memory. When an engine starts with --load-format ipc_cache, it builds the model skeleton on PyTorch's meta device — shapes only, no memory — then asks the daemon over a Unix socket for its tensors. The daemon answers with CUDA IPC handles, and the engine drops each handle into the matching parameter slot. In the default zero_copy mode, no weight bytes move at all: the engine computes on the daemon's memory directly.

The stagehand only lets a cast in if it is performing the same play. Before the engine accepts the cached weights, it compares a fingerprint with the daemon's, and any difference sends it back to the warehouse. The fingerprint covers a hash of the checkpoint's safetensors metadata (the descriptive header stored at the start of each weight file), the model architecture, the TP size and this GPU's rank (its position among the TP GPUs), the data type, the quantization method and a hash of its config, the model revision, and the vLLM version. A mismatch, or no daemon at all, falls back to an ordinary disk load unless you set fallback to false. A few details keep the shortcut honest: the daemon only opens its socket once the whole model is cached, so an engine never maps a half-built set; the socket lives in an owner-only directory and both sides verify that the socket and its directory belong to the current user and are not open to other users; and a quantization method that has not been verified for sharing fails when the daemon starts, not halfway through a load.

There is also a copy mode. Instead of sharing the daemon's memory, the engine clones each tensor into memory it owns and then tells the daemon to release its copy. That costs one on-GPU copy per restart but leaves the engine fully independent of the daemon afterwards. The trade-off matters because zero-copy weights belong to the daemon's allocations — which is why the loader's documentation says vLLM's sleep mode, which offloads weights to free GPU memory, must not be used with it.

Work one restart through with the model the pull request itself uses as its example: Qwen3.5-122B-A10B in FP8, on four GPUs. Hold three numbers fixed. At FP8's 1 byte per parameter, 122 billion parameters are roughly 122 GB of weights, so each of the four GPUs holds about 30.5 GB. Assume the checkpoint streams from shared storage at 2 GB/s in total (illustrative). A disk load then spends about 122 ÷ 2 ≈ 61 seconds just reading bytes, before any repacking — on every restart. In zero_copy mode the restarted engine moves 0 GB of weights: it opens handles to memory that is already in place. Even copy mode only clones 30.5 GB inside each GPU's own memory, which at an illustrative 3 TB/s of memory bandwidth — 61 GB of reads plus writes — is about 20 milliseconds. The work of the first 61 seconds does not shrink; it moves to the daemon and is paid once instead of on every restart.

Two limits follow directly from how the fingerprint is built. Fast Start helps the restarts where the weights are unchanged — recovering from a crash, or changing a serving setting such as the maximum context length — and not a vLLM upgrade, because the vLLM version is part of the fingerprint, so a new version misses the cache and loads from disk. It also leaves the rest of startup alone: unless the engine runs with --enforce-eager, CUDA graphs are still captured, and graph capture alone can add tens of seconds. The same release attacks that second cost with a separate change — freezing Python's garbage collector during capture — which the release notes credit with cutting capture from 12 s to 2 s and engine initialization from 28.9 s to 8.2 s on H200. Those two figures belong to that change, not to Fast Start, which the release notes do not give a timing for.

Goes deeper in: LLM Serving → CUDA Graphs → Production Tradeoffs

Related explainers

• AMD serves NVFP4 checkpoints on MXFP4-only GPUs — an example of the load-time weight conversion that Fast Start pays once instead of on every start.
• Miles — rollout-trainer separation with asynchronous weight sync — the other direction of the same problem: getting new weights into running engines quickly.
• vLLM 0.28 — partial loads from a lower cache tier — the same engine keeping KV cache, rather than weights, somewhere it can reach cheaply.