Open-source Miles for asynchronous post-training across 64 GB300 GPUs — Rollout-trainer separation with asynchronous weight sync — What does it mean?

The news. On September 8, 2026, the Miles team published Miles v0.1: Production-Level Post-Training, an open-source post-training system and a report that walks the whole reinforcement-learning loop end to end: rollout engines built on SGLang, a trainer with a choice of two backends (NVIDIA Megatron-LM and PyTorch FSDP), and three weight-synchronization transports for different deployment topologies. The closing case study runs fully asynchronous agentic RL on a GLM-5.2 744B-A40B model over terminal-use coding tasks, on 64 NVIDIA GB300 GPUs, with a median step time of 263 seconds over the first 30 measured steps. Read the report →

Picture the company mid-run. The cast is on stage, each actor playing from the page in their hand, and next door the writers are rewriting those pages based on how the last scenes went. Nothing about that arrangement is decorative. The two halves of RL post-training run different shapes of work, and building them as two systems is what lets you place and schedule each one on its own terms. Generating a trajectory is a serving workload: one token at a time, limited by how fast weights can be read out of memory. Computing the update is a training workload: large matrix multiplications, limited by arithmetic throughput. That is the prefill-versus-decode split one level up: the same two bottlenecks, now sitting in two different programs. Miles gives the first job to SGLang rollout engines — the same kind of engine that serves your production traffic — and the second to a Megatron-LM or FSDP trainer.

The handoff between them follows a single contract, and every other mechanism in the system is an optimization of it: the trainer prepares new weights, transfers them to every rollout engine, and only then do the engines start generating under the updated policy. Miles runs that handoff on a configurable cadence, after every training step by default. In the theatre that is the writers finishing a draft and the pages reaching every actor: the next scene is played from the new draft, while a scene already under way may finish on the old one. Hold on to that second half — it is where the interesting cost turns up later.

Where you put the two systems decides whether they can run at the same time. In a colocated placement they share the same GPUs, so they take turns: the engine generates, then the training actor — the process holding the weights, gradients and optimizer state — is brought back onto the card to update. Miles even evicts the paused training actor from GPU memory for exactly the interval in which the engine is generating. In a disaggregated placement they sit on separate GPU pools and run concurrently, which is the only way to overlap them — and the reason Miles refuses to start a fully asynchronous run when the trainer and the rollout engines share GPUs.

That overlap is not bonus performance; it is recovered waste. In a synchronous, turn-taking schedule a batch is done only when its slowest trajectory is done, so most of the fleet sits idle waiting on one long tool-using episode, and the trainer waits too, because it needs the complete batch before it can do anything. Serving engineers already have a name for that shape of loss: capacity that is allocated and paid for but is not producing anything anyone can use — the gap between throughput and goodput.

Now the step that is easiest to underestimate. Moving the weights is not bookkeeping between the interesting parts — at frontier scale the transfer can dominate the step. The report's own illustration of the problem: a full NCCL broadcast of Kimi K2, a 1T-A32B model, takes almost a minute. Miles works on that from two directions. First, preparation. Rather than calling the rollout engines once for each of a large model's many thousands of tensors, the Megatron backend's preparation pipeline gathers the pieces of each tensor that the trainer's GPUs hold separately — its tensor-parallel shards — converts them into the names and layout the inference engine expects, and appends them to a fixed-size buffer — 512 MB by default — which is handed to the transport as one bucket when it fills. (The FSDP backend takes a simpler path: it buckets its state dictionary and reaches rollout engines by broadcast alone.) Second, the transport itself, which the report treats as a choice rather than a constant.

Read that table as a statement about your network, not about your model: all three deliver the same converted weights, and what separates them is the connectivity they assume and how the transfer volume grows as the fleet grows. Broadcast and peer-to-peer differ only in how a filled bucket leaves the trainer — one batch of NCCL broadcasts, or RDMA writes into the memory of the rollout ranks, one rank being one process of the distributed job — which is the same NVLink-versus-network question that decides how a KV cache moves between prefill and decode pools. Disk-delta changes what is shipped rather than how: every rollout host starts from a shared base checkpoint, the trainer writes out only the bytes that changed plus a reference to the base they apply to, and each host patches its own local copy. It also comes with the sharpest restrictions — Megatron backend only, and Miles refuses to combine it with colocation, LoRA (low-rank adapter) runs, or prefill/decode disaggregation.

One detail deserves copying into your own systems. A transport that silently drops a tensor would be invisible: the engine keeps generating, slightly wrong, forever. Miles closes that hole in two separate places. Once, at the start of training, it fills every engine tensor with random values before running the first update and then compares each engine against the trainer, so a tensor the transport never wrote still holds that noise when the check reaches it and cannot pass unnoticed. And on a disk-delta update, a failed write or a failed checksum stops the update before any engine reloads. The rule both mechanisms protect is the same: no engine ever generates from a half-applied set of weights.

Overlapping the two systems has one unavoidable consequence: some finished trajectories were produced under weights the trainer has already moved past. Miles does not pretend otherwise; it bounds the problem in the data buffer, which will retry or discard a finished group — the set of attempts at one prompt that the trainer collects together — for one of three reasons. Generation gave up on it: an agentic episode ran past its collection timeout and never completed. A user filter rejects it: every attempt at that prompt earned the same reward, so the group carries no signal about which attempt was better, and that one is discarded outright. Or its weights are simply too old.

The third check is the one with a subtlety worth understanding. The first two are fixed properties of the group, so they are checked the moment it arrives. Staleness is not: a group keeps getting staler while it waits in the buffer, because the trainer goes on updating. So staleness is checked on the way out, when the trainer actually collects the group. A group can also arrive stale to begin with, when its turns spanned several weight updates during generation — which a long, multi-turn agentic episode is especially exposed to.

Put numbers on the transport choice, holding the model and the step fixed. At BF16 — two bytes per parameter — 744B parameters is about 1.5 TB of weights (744 × 10⁹ × 2 bytes). Broadcast ships all of it on every update, and the default cadence is one update per training step. Disk-delta ships only the bytes that changed; the report says consecutive RL steps change only a small fraction of a model's bytes, but it does not give that fraction, so take 2% as an illustrative figure: at that illustrative 2%, about 30 GB in place of 1.5 TB, roughly 50× less to move. For a sense of what the difference is worth, set the report's own anchor (a full NCCL broadcast of a 1T-A32B model takes almost a minute) beside the case study's 263-second median step: as an illustration only — the two numbers describe different models, and this is a measurement of neither run — a sync of that size would be a fifth to a quarter of a 263-second step, spent copying rather than computing. The direction is the point. On a fleet where the copy is that expensive, the transport is not a configuration detail.

Goes deeper in: LLM Serving → Prefill/Decode Disaggregation → Full Disaggregation

Related explainers

• SIS — Selective Importance Sampling — what to do with the off-policy tokens a stale rollout hands you, instead of throwing the group away.
• CacheRL — Cached rollouts for agent RL — the other lever on the same cost: reuse rollouts rather than regenerate them.