NVIDIA open-sources Snapshot — Checkpoint-restore vs cold start for inference replicas — What does it mean?

The news. On November 10, 2026, at KubeCon, NVIDIA announced it is contributing its OpenShell agent-security runtime to the CNCF and released several open-source Kubernetes components, one of them NVIDIA Snapshot. Snapshot checkpoints a fully initialized GPU pod (the Kubernetes unit of running containers) and restores it on a node with matching GPU hardware and driver. In partner tests, Crusoe cut GLM-5.3 readiness from roughly eight minutes to 31 seconds, and Saturn Cloud cut vLLM scale-from-zero startup from 2–8.5 minutes to 6–34 seconds. Read the announcement → · Snapshot repository →

Picture an old console game that makes you sit through the boot screen and replay the same tutorial levels every time you switch it on, before it lets you play. An inference server pays the same toll on every launch, and it pays it again for every new replica. First comes the boot: Python imports and the worker processes that vllm serve actually spawns. Then the cartridge goes in: the model load reads the weights from storage into GPU memory (HBM). Then setup: the engine profiles memory, works out how much KV cache fits, then allocates it. The longest level comes last: compiling and autotuning kernels, then capturing a CUDA graph for every batch size on the capture ladder — the four stages vLLM's compile_or_warm_up_model runs before it accepts a single request.

In NVIDIA's measurements, loading the weights is not the slow part. Across all 22 configurations in Snapshot's benchmark — vLLM, SGLang and TensorRT-LLM on one to eight B200 GPUs — model load was 3–25% of the cold start, while the combined compilation, autotuning and graph-capture stage was 33–82%. A faster disk only shortens the cartridge step. Those runs started each cold start with empty compiler caches, so a server that keeps its compile cache between launches spends less there; but graph capture still runs in every new process, which is why the CUDA graphs tradeoff has always been "slow start, fast decode" or --enforce-eager's "fast start, slower decode".

An emulator save state does not record how you reached level 5. It copies the whole machine's memory at that moment, so loading it puts you back at level 5 at once. Snapshot does the same to a ready inference process: it saves the process's CPU memory and GPU memory plus the container's file changes, and a restore hands back a process that has already loaded, compiled and captured. CRIU saves and rebuilds the process tree on the CPU side; cuda-checkpoint, or Snapshot's PageBroker GPU engine, handles GPU memory. The weights stay resident in GPU memory, so they come back inside the memory image rather than through the model loader.

You put the controller down before you save. The workload signals that no GPU work is in flight before Snapshot takes the checkpoint, and in the benchmark each engine was first asked to release its KV cache memory, so the image does not carry a large, empty KV pool; the wake call at the end of a restore brings the engine back to serving. One detail surprises people: taking the checkpoint ends the source process, with no resume-after-checkpoint mode, so the usual pattern is a dedicated job (Snapshot calls it a SnapshotJob) that starts a replica, waits until it is ready, and saves it once.

All three rows come from NVIDIA's own runs on B200 GPUs with vLLM 0.31.0, fresh compiler caches, and weights already on shared network storage (NFS); both times stop when the engine is ready, before the first request. Notice the Snapshot column: the image is larger than the weights, because it also holds the rest of the process's CPU and GPU state and the container's file changes, so restore speed now depends on how fast the snapshot store can be read.

Hold three things fixed: DeepSeek-V4.1-Flash, vLLM, and one eight-GPU B200 node. A cold start takes 1,110.65 s, about 18.5 minutes, and 784.54 s of it (about 71%) is compilation, autotuning and graph capture, against 184.90 s for reading 510 GB of weights. A restore takes 30.96 s, of which 19.37 s is putting GPU memory back — the 35.87× speedup. Now suppose (illustrative) a traffic spike makes the autoscaler add one replica: with a cold start, eight paid-for GPUs serve nothing for about 18.5 minutes; with a restore, for about half a minute. The checkpoint itself took 274.75 s, but it is paid once, and every later replica restores from the same image, so the trade is one ~4.6-minute save against ~18 minutes saved on each replica after it.

A save state only loads on the same console and firmware, and a Snapshot image only restores on a node with matching GPU hardware and driver version. The current release also has hard edges: a workload must fit on one node (multi-node is on the roadmap), vGPU is not supported, a pod gets exactly one restore, and tools that intercept libcuda.so (CUDA driver library) calls, such as some GPU monitoring agents, can make CUDA calls fail or hang after restore. NVIDIA says the APIs may still change and does not yet recommend Snapshot for production-critical workloads.

Where it earns its keep is capacity planning. The minutes a new replica needs to become ready are minutes of headroom you must already be holding when a spike starts; cut readiness from minutes to seconds and an autoscaler can respond to a spike soon after it starts instead of holding headroom ahead of it, which makes keeping idle models at zero replicas more practical.

Goes deeper in: Inside vLLM → Compilation & CUDA Graphs → What Startup Buys You