Apple's Minions GPU cloud — Pull-based work claiming — What does it mean?
The news. On October 9, 2026, Apple researchers posted Minions, a "dynamic and distributed virtual GPU cloud" built from idle Apple Silicon Macs. Teams submit embedding-generation or LLM-completion jobs; a server splits each job into subtasks in PostgreSQL; executors on Macs pull subtasks, run them on the local GPU (Hugging Face transformers for embeddings, Ollama for completions), and write results to Amazon S3. The paper reports that jobs which used to wait up to 10+ hours for the central GPU cluster could often start within minutes, with no new GPU requests. Read the paper →
Picture a busy kitchen with no one assigning orders. Every order is a paper ticket clipped to a rail. A cook who finishes a dish walks to the rail and takes the next ticket; a cook who is on a break simply does not take one. Nobody has to keep a list of which cooks are present, because presence shows up as the act of taking work. That is the whole idea of pull-based execution: the worker asks, the queue answers.
Now compare this with the kitchen most systems are built like, where a head chef reads each ticket and hands it to a named cook. The head chef is a push scheduler. It can make smarter choices (give the long order to the fastest cook), and inside one GPU that is exactly what you want: the scheduler in an inference engine owns its GPU and plans every batch. But a push scheduler is only as good as its picture of the workers. When the workers are personal Macs, that picture is wrong all the time: a laptop lid closes, Wi-Fi drops, the owner needs the machine back. Minions removes the head chef instead of trying to keep its picture current.
Two kitchen rules make this safe. The first is one ticket per spike. Two cooks can reach for the same ticket at the same moment, so something must decide who gets it. Minions uses the database itself: each executor tries to INSERT an ownership row into a table called minion_subtask_acquirement, and a unique constraint on (team, project, task name, subtask id) lets exactly one insert succeed. The losers see a uniqueness violation (or a skipped ON CONFLICT DO NOTHING) and go pick a different subtask. No lock server, no leader election: the database's existing guarantee is the referee.
The second rule is calling out. A cook holding a ticket shouts "still on it" now and then, and an expediter watches for cooks who go quiet. In Minions each running executor sends a heartbeat every five minutes, which the server writes to a heartbeat table. The watchdog checks heartbeats only for executors that currently hold a subtask, not every registered Mac, and when one goes silent it releases that subtask back to the queue for a healthy executor to claim. The paper does not state how long the watchdog waits after the last heartbeat before it reclaims a subtask.
Here is where the time actually goes, with illustrative numbers (the paper reports no job-level timings). Take a job of 1,200 subtasks of 30 seconds each on 20 Macs. Pulling one subtask at a time, the pool finishes in about 1,200 × 0.5 min ÷ 20 = 30 minutes. Now 5 Macs fall asleep at minute 10. By then about 400 subtasks are done, so roughly 800 remain for 15 Macs: 800 × 0.5 ÷ 15 ≈ 27 more minutes, so the job ends near minute 37 instead of 30. What the 5 sleeping Macs took with them is only the 5 subtasks they were holding, and even if the watchdog needs 15 minutes (illustrative) to notice, those 5 return to the queue around minute 25, well before it drains. Compare a push scheduler that pre-assigned 60 subtasks to each Mac at the start: at minute 10 each sleeping Mac still has 40 unstarted subtasks on its list, so 200 subtasks are stuck, not 5, until the scheduler detects the failures and re-plans.
| Metric | Before Minions | With Minions | Source |
|---|---|---|---|
| GPU queue time | Hours (up to 10+) | Minutes | Table 1 |
| New GPU requests | Required | None | Table 1 |
| Infrastructure cost | High | Zero (as reported) | Table 1 |
| Scaling effort | Manual | Organic | Table 1 |
Pull-based claiming has costs, and they are worth naming. The default order is plain first-in, first-out, so a pull queue cannot place work cleverly the way a push scheduler can. Minions adds a small escape hatch called dynamic task binding: a Mac's owner can tell its executor to prefer one team, project or task, and the server turns that preference into a database filter; when nothing matching is pending, the executor falls back to the global queue. Second, reclaiming work from a silent worker means a task can run twice if that worker was only slow, not dead. The paper does not describe how a late result is handled, so treat the general rule as an assumption to check: a requeue design gives at-least-once execution, and outputs should be safe to write twice. Third, capacity is social: the paper notes teams often recruit executors with a Slack message, which is the capacity planning problem solved by asking colleagues instead of buying GPUs.
The contrast with a tightly coupled job is the useful mental model. In vLLM tensor-parallel serving, one dead worker process shuts down the executor of its whole replica, because each worker holds a slice of every layer and the others cannot finish a forward pass without it. Minions works because embedding and completion subtasks are independent: no subtask needs another one's result, so losing a worker loses one subtask, not the job.
Goes deeper in: LLM Serving → Inference Engine → The Scheduler