Sandman paper — Pooled persistent sandboxes for agent RL — What does it mean?
The news. On October 29, 2026, an Amazon team posted the Sandman paper. It reports a scale test that held 11,858 concurrent tiny sandboxes and four SWE-bench Verified training campaigns that created 3.1 million sandboxes during October 2026, and it covers burst handling, environment preparation and abandoned-sandbox cleanup. Read the paper →
Agentic reinforcement learning (RL) trains a model by letting it act, such as running commands, editing files and running tests, and then scoring the result. Every attempt is a rollout, and every rollout needs its own sandbox that remembers its state between tool calls. Sandman supplies that sandbox as a Kubernetes pod running the task's unmodified image plus the small execd daemon.
Picture the hotel front desk. A guest arrives and expects a room that keeps their luggage while they walk to the lobby and back. The desk has three ways to supply the room, and each one trades the guest's wait against the hotel's bill for empty rooms. It can build the room on arrival, keep rooms made up in advance, or send housekeeping to reset a used one.
On demand is the default, and in the paper's measurements its visible cost is the image pull. In the paper's re-grading study on SWE-Gym images of 1.3–8.4 GB, a create that had to pull took a 17.4 s median (p90 74.3 s), against 4.1 s (p90 5.1 s) when the node already held the image. The room is quick to prepare once the furniture is in the building; hauling the furniture in is what keeps the guest waiting. A fixed 120 s readiness budget that included the pull rejected 9 admissible tasks of a 280-task set, so the paper moved to a per-job budget.
A warm pool removes that wait by making rooms up early. The pool keeps never-used pods per (image, base commit), hands one out only after a health check, and starts a replacement whenever it does. The price is that a warm pod holds its full CPU and memory request while it idles, and the paper notes that requests equal limits, so idle time between tool calls is a cost it chose not to overcommit away.
| Option | Hotel picture | What it buys (paper-reported) | What it costs (paper-reported) |
|---|---|---|---|
| On demand | Build the room on arrival | Fewest pod-hours in all 8 replayed traces of the pre-registered comparison | Median create 17.4 s with a pull vs 4.1 s without (SWE-Gym images) |
| Warm pool | Rooms made up in advance | Grading acquisition 6.2 s → 0.12 s; 93–96% of gradings served warm (3 run pairs) | 2.46× the sandbox vCPU-hours at a 2,100 s idle timeout (setup-dependent, illustrative of that run) |
| Reuse | Housekeeping resets a used room | 29% fewer creates per grading than a warm pool (95% CI 22–36%); pods reused at most 8 times | 1.21 creates per grading vs 1.00 on demand in the pre-registered setup; needs an exact-reset baseline |
A 50-fold faster acquisition bought only a few points of waiting time, at 2.46 times the compute bill. Hold three paper numbers fixed. Grading acquisition fell from 6.2 s to 0.12 s (means of per-run medians). The pool served 93–96% of gradings warm. Sandbox vCPU-hours rose 2.46× at a 2,100 s idle timeout. Take 1,000 gradings (illustrative): 1,000 × (6.2 − 0.12) s = 6,080 s, about 101 minutes of acquisition wait removed in total. Yet the learners' waiting share fell only from 7.0% to 4.2%, and time to step 24 improved by 9.4% (95% CI 5.9–13.0%). In replays, a 300 s idle timeout kept the warm-hit rate at 94.5–97.7% at about half the idle cost, so the timeout, not the pool itself, is the knob that sets the ratio.
The failure that costs most is not slowness but pods nobody owns. A trainer may abandon an acquisition after a timeout even though the Kubernetes API server has already accepted the pod. If the cancellation also cancels the caller's cleanup, the pod becomes ready with nobody waiting, and the retry creates another one; in a stress test this left at least 4,181 excess pods. In hotel terms, the guest cancelled after the room was built, and the desk never cleared it.
Sandman's fix is that the provider owns each create until it settles: the create runs even if its caller is cancelled, cleanup runs on any exit, and a nonce on each create lets the provider find a pod the API stored but answered with an error. Unconfirmed deletes are retried, and an owner reference lets Kubernetes remove what a closed provider could not. This is idempotency applied to infrastructure: a retry must not duplicate the side effect.
The paper does not settle everything. The warm pool's steady-state effect on step interval is unresolved: the mean difference was −8.7 s with a 95% CI of −38.4 to +21.0, and three run pairs can only detect changes above 24% of the step interval. How often production creates actually pulled an image was not measured. Reuse beat the warm pool on creates but not on pod-hours, where on demand stayed lowest, and the authors did not test replacing only the pods that are not returned. These are results from one organization's workloads, so size your own pool from your own request rate and create time.
Goes deeper in: Agent Engineering → Production Harness Architecture → Idempotency — Safe to Re-run