What is a mean-field contention surrogate?

It is a learned predictor that estimates how fast a model will run on a shared GPU from two inputs: that model's own configuration, and a compact set of features summarising aggregate GPU state. The mean-field part is that it deliberately does not represent which specific models are co-running, replacing those neighbours with an averaged summary of the load, the same simplification physics uses to avoid tracking every pairwise interaction. The payoff is in what it costs to build: the samples needed grow approximately linearly with the number of concurrent models rather than combinatorially.

Why does profiling colocated models get expensive so quickly?

Because contention is not a fixed tax you can measure once. How much a model slows down depends on which other models share the GPU with it, so a predictor built on joint interactions has to be trained across those combinations. Each extra model you allow onto the device multiplies the number of combinations to measure rather than adding to it, so the offline profiling bill runs out of reach well before the number of colocated models gets large.

How does the surrogate relate to the scheduler that uses it?

The surrogate only produces estimates; a genetic-algorithm scheduler is what turns them into placements. It keeps a population of candidate placements and repeatedly mixes and mutates the promising ones, which is only practical for an online scheduler when scoring a candidate is cheap, and that is what the surrogate provides. The paper reports that on a 5-model problem with 78,732 feasible joint configurations this pairing lands within 0.10% of exhaustive search, with zero SLA violations across eight dynamic scenarios and a median decision time of 26 ms.

GPU· 2026-09-04

MeanField surrogate schedules concurrent models on a shared GPU — Mean-field contention surrogate — What does it mean?

TL;DR

What is it: The news anchor is a paper on MeanField surrogate modeling for scheduling several AI models that run at the same time on one GPU; the concept it makes concrete is a mean-field contention surrogate, which predicts each model's speed from its own settings plus a compact summary of aggregate GPU state instead of from which models it happens to be sharing with.
Why it’s needed: Putting several models on one GPU is how you stop paying for idle hardware, but they slow each other down, and a scheduler cannot honour a latency promise it cannot predict, so the predictor has to be both accurate and cheap enough to build.
vs previous: A surrogate that models joint interactions has to cover the combinations of co-running models, so its profiling cost grows combinatorially; the mean-field surrogate replaces the identity of the neighbours with a summary of aggregate GPU state, and its sample budget grows roughly linearly in the number of models.

Jargon

Colocation: Running more than one model at the same time on the same GPU, so the hardware is not idle between one model's requests. The cost is that those models then compete for the same compute units and the same memory bandwidth.
Contention: The slowdown each colocated model causes the others by taking a share of a resource they all need. It is not a fixed tax: the same model can run near full speed beside a light neighbour and much slower beside a heavy one.
Surrogate model: A small learned predictor that stands in for actually running the thing. Here it takes a proposed placement and returns predicted performance, so a scheduler can evaluate thousands of options without launching any of them.
Joint profiling: Building that predictor by measuring the specific combinations of models running together. It is the accurate way and the expensive way, because the number of combinations to measure multiplies as you allow more models onto the GPU.
Mean-field approximation: A method borrowed from physics: instead of tracking every pairwise interaction, replace the neighbours with an averaged summary of their combined effect. Here that summary is aggregate GPU state.
Genetic-algorithm scheduler: A search method that keeps a population of candidate placements, then mixes and mutates the promising ones and repeats. It is only practical when scoring a candidate is cheap, which is what the surrogate provides.
SLA violation: A request that misses the service-level target it was promised. The paper's scheduler searches under service-level constraints and reports zero violations across its eight scenarios. See SLO-driven system design.
R² (coefficient of determination): How much of the real variation in performance a predictor accounts for, on a scale where 1.0 is perfect. The paper reports approximately 0.96 across 2 to 6 concurrent workloads.

The news. On September 2, 2026, a paper proposed a MeanField surrogate for scheduling concurrent, heterogeneous AI inference on a shared GPU. Its argument is that surrogate models already avoid costly online benchmarking, but their profiling requirements still grow combinatorially with the number of co-running models. It instead predicts each model's performance from local configuration plus aggregate GPU state, reports R² of approximately 0.96 across 2 to 6 concurrent workloads, and drives a genetic-algorithm scheduler with it. Read the paper →

The app does not know which cars are on your road

A traffic app tells you the drive will take 25 minutes without knowing which particular cars are out there. It reads one number for how congested the city is, combines that with your own route, and hands you an estimate good enough to plan around. Timing the actual journey for every possible set of cars that could share the road with you would be exact, and also hopeless, because the number of possible sets multiplies every time another car pulls out of a driveway.

A GPU running several models at once is in that position. Colocation is how you stop paying for idle hardware, but the models contend for the same compute units and the same memory bandwidth, so each one runs slower than it would alone, and how much slower depends on which models it is sharing with. A scheduler placing an incoming vision model needs that answer before it commits, and needs it fast enough not to stall the placement, so measuring the actual combination first is not a real option.

The standard answer is a surrogate: a small learned predictor that takes a proposed placement and returns predicted performance, letting the scheduler weigh options without launching any of them. The difficulty is in training it. A surrogate that represents joint interactions has to be trained across those combinations, and that is the same multiplying set the traffic app declined to enumerate. So the surrogate removes the online benchmarking and leaves behind an offline profiling bill that still grows combinatorially with the number of models you are willing to colocate.

It is also why a placement decision cannot afford to be casually wrong about load. Queueing does not give way gently: push utilization up and latency holds, then goes all at once, as the dial below shows. Being slightly wrong about how loaded the device already is can land you well past that knee.

L=λ×W

L5in-flight= computed

λ50req/s

W0.10seconds

Derived:

λ50 req/s

Little's Law: L = λ × W is exact for any stable queue. Service time fixed at 50 ms. As λ approaches capacity (~100 req/s), W diverges — that's the knee in the saturation chart on the right.

Replace the neighbours with a summary of the load

The paper's move is to stop modelling who the neighbours are and model only how much total pressure they put on the GPU. Each model gets a predictor over two inputs: its own local configuration, and a compact set of mean-field features describing aggregate GPU state. This is the simplification physics reaches for when a particle is treated as sitting in one averaged field rather than in a web of pairwise forces, and it buys the same thing here. The identity of the co-runners drops out of the input, so the number of samples you must collect grows approximately linearly in the number of concurrent models rather than combinatorially. The paper reports this holding at R² of approximately 0.96 across 2 to 6 concurrent workloads.

What the swap is worth

A toy illustration follows. These are not the paper's numbers and not its profiling procedure; it is here only to show multiplicative growth against additive growth.

Hold two quantities still and the shape of the saving is easy to read. Suppose each model has 8 candidate configurations and you want to serve 5 models at once. Profiling the joint configuration means measuring every combination of those choices: 8 x 8 x 8 x 8 x 8 = 32,768 measurements. Do it the mean-field way instead: bucket aggregate GPU load into 8 levels, and each model needs its 8 configurations measured against those 8 levels, so 64 measurements per model and 320 across all five. Adding a sixth model multiplies the joint approach by 8, to 262,144; it costs the mean-field approach another 64. That gap between multiplying and adding is the shape of the paper's actual claim: a sample budget that grows approximately linearly in the number of models, where fully joint profiling grows combinatorially.

An estimate is only worth what acts on it

The paper pairs the surrogate with a genetic-algorithm scheduler, which keeps a population of candidate placements and repeatedly mixes and mutates the good ones, which is only practical for an online scheduler when scoring a candidate is cheap. The paper reports that on a 5-model problem with 78,732 feasible joint configurations, that search lands within 0.10% of what exhaustive search finds, with zero SLA violations across eight dynamic workload scenarios, and a full online decision takes a median of 26 ms, about 5x faster than searching the surrogate exhaustively. The traffic-app comparison holds to the end: the app is not computing the one true fastest route from complete knowledge of every car, it is testing many plausible routes against a good-enough congestion estimate, fast enough that the answer still matters when it arrives.

Predicting colocated performance	Joint-interaction surrogate	MeanField surrogate
What each model is predicted against	the specific set of co-running models	its own configuration plus aggregate GPU state
How the profiling budget grows with N	combinatorially	approximately linearly
Reported predictive accuracy	R² of approximately 0.96 for the MeanField surrogate, across N = 2 to 6
Reported scheduling result at N = 5	within 0.10% of exhaustive over 78,732 feasible configurations, zero SLA violations in 8 scenarios, 26 ms median decision

What the approximation actually costs

What makes this read as a scheduling result rather than a modelling one is where the approximation is spent. Dropping the co-runners' identity from the input plausibly costs you the ability to say that model A specifically is what is slowing model B down, and buys back a predictor you can afford to build at this scale — a reading the paper invites rather than states. The trade would pay because a scheduler does not need the attribution: it needs a number good enough to rank placements under a service-level target, and ranking tolerates an approximation that attribution would not.

Goes deeper in: LLM Serving → The Inference Engine → The Scheduler

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based