Route local agent inference across mixed devices with NVIDIA PAIR — Request-level routing — What does it mean?
The news. On September 3, 2026, NVIDIA released PAIR as an open beta for Windows, macOS and Linux. Install it on each machine, approve a pairing request, and your agent harness points at PAIR instead of at one laptop. It works with Ollama and LM Studio on GeForce RTX 20 Series and newer, RTX PRO workstation GPUs from Turing onward, DGX Spark, and Apple M4 or newer silicon. In NVIDIA's own five-subagent demo running Qwen 3.6 35B A3B, a three-device set finished in 8 minutes 48 seconds against 18 minutes on a single RTX Spark laptop. NVIDIA labels that demo configuration-specific, and says plainly that it is not a general benchmark or a promise of linear scaling. Read the release →
Picture a salon with three chairs. A client walks in and the receptionist cuts nobody's hair. She looks down the row and asks four things: is this stylist in today, are they free right now, do they know this particular cut, and how many people are already waiting on them. Then she points at one chair. PAIR's proxy is that receptionist, and its real filter list is almost literally those four questions — whether a paired node is online and ready, whether a supported inference engine is enabled, whether the exact requested model is present, the node's current workload including active jobs, and how much of its GPU is already spoken for by something else on screen, such as a game or a 3D tool.
What surprises people is what happens next. The client does not move between chairs. Once seated they stay put until the cut is done, and NVIDIA states the same rule about requests in one sentence: "Every request is assigned to one eligible node and remains there for its lifetime." PAIR does not merge GPUs, pool their VRAM (the memory on the graphics card itself), or shard a model. Every machine in the set holds the whole model on its own, and the router only picks which whole copy runs your call.
Getting the chairs into one salon is the unglamorous, load-bearing part. PAIR finds nearby machines with mDNS, you approve each pairing by hand, and from that moment every node-to-node message travels over mTLS with certificates PAIR generates. Until a pairing is approved, node-to-node traffic is blocked outright — the salon has a door, and the receptionist decides who is behind it.
So why can the receptionist not put half a haircut in each chair? Because generation is sequential. The model emits one token at a time, and each new token is computed from every token before it, held in a KV cache that lives in that one GPU's memory. Splitting a single request across two machines would mean shipping that cache back and forth over your home network between every token, which is why the techniques that do split one request, tensor parallelism among them, assume NVLink-class bandwidth inside a single box rather than Wi-Fi between rooms. PAIR sidesteps the problem by never splitting anything: it raises how many requests you can run at once, which is a different quantity from how fast any one of them finishes.
That makes PAIR a scheduler one level up from the one you may already know. The scheduler inside a single serving engine decides which of the requests already on this GPU get to run in the next batch; PAIR decides which GPU sees the request at all. Both are choosing an order under a constraint, and neither makes an individual request compute any faster.
| Approach | What each device holds | What actually gets faster | What it needs |
|---|---|---|---|
| PAIR request routing | the whole model, one copy per device | throughput — ~2.05× on NVIDIA's three-device demo (configuration-specific, not linear scaling) | an ordinary local network |
| Tensor parallelism | a slice of every layer | one request, and a model too big for one GPU now fits (varies by model and interconnect) | a fast interconnect inside one machine |
| A single device | the whole model | nothing — independent jobs queue behind each other | one machine |
Hold the demo's shape fixed: five subagent jobs, three devices, one laptop as the baseline. NVIDIA measured 18 minutes on the single RTX Spark laptop and 8 minutes 48 seconds, which is 8.8 minutes, on the three-device set. That is a 2.05× speedup, not 3×, and the shortfall is arithmetic before it is overhead. Five jobs across three chairs cannot all be seated at once: three go first, two go second. So even if every job cost the same and all three machines were equally fast — two idealizations, since the set mixes a laptop, a DGX Spark and an RTX 5090 — a round-based router still needs two rounds to clear five jobs' worth of work, which caps it at 2/5 × 18 = 7.2 minutes, a 2.5× ceiling (treating the laptop's 18 minutes as five jobs of serial work; illustrative). The measured 8.8 minutes sits 1.6 minutes above that ceiling, and the second idealization is the likeliest reason: a round ends when its slowest machine ends, so the laptop sets the pace for whichever round it is in. No single job got faster because it was routed — a job only finished sooner if it happened to land on a faster machine.
Goes deeper in: LLM Serving → Inference Engine → The Scheduler
Related explainers
- AMD Atom: prefill/decode disaggregation — the other way to spend two machines on one workload: split a request's phases across them instead of keeping it whole.
- SGLang v0.5.14: LPLB load balancing — the same balancing question one level down, spreading load across experts inside a single cluster.