What is correctness-gated assembly optimization?

It is an optimization loop that edits a GPU kernel at the assembly level and admits a candidate only after checking that it still behaves like the original on identical launches. In AsmEvo, a reassemblable representation is recovered from the compiled AMDGPU code object, an agent proposes low-level edits, the result is rebuilt as an ABI-preserving object, and differential verification against the original under identical launches decides whether the candidate is accepted. Timing happens only after that gate passes, so speed ranks the survivors rather than selecting them.

Why edit assembly instead of the Triton or CUDA source?

Because the source is not always the thing that runs. A deployed kernel can arrive as a binary whose source is unavailable, or whose source sits several compiler passes away from the machine code that executes — so the paper treats the deployed binary as the reference for the kernel's behaviour. Working at that level keeps the optimization on the artifact that is actually in production, at the cost of giving up readable source and taking on the burden of proving equivalence yourself.

How much speedup does AsmEvo report, and on what?

On a selected KernelBench subset on MI308X the paper reports improvements in 29 of 30 kernels, with a 1.35x geometric-mean and 3.88x maximum speedup. On MI300X it reports 1.09x geometric-mean and 1.31x maximum on AMD's AITer binaries, and 1.18x geometric-mean and 1.34x maximum on vLLM/SGLang Triton assembly kernels, in each case reported as preserving functional equivalence. Across those three, the smallest gains are on AMD's own AITer binaries.

AsmEvo tunes AMD GPU kernels at the assembly level

TL;DR

What is it: AsmEvo is a kernel-optimization system, published as a paper on August 21, 2026, in which a long-horizon agent — one that keeps working across many edit-and-test rounds rather than answering once — rewrites the AMDGPU assembly of an already-compiled kernel and a verifier decides which rewrites are allowed to survive — the idea it names is correctness-gated assembly optimization.
Why it’s needed: Production GPU kernels often reach you as binaries whose source is unavailable, or several compiler passes away from the machine code that actually runs, so the last slice of performance sits at a level the usual Triton or CUDA workflow rarely reaches.
vs previous: Source-level tuning edits Triton or CUDA C++ and trusts the compiler to carry the intent all the way down; AsmEvo edits the shipped object directly and replaces that trust with a differential check against the original under identical launches, so an edit counts as a speedup only once its output has been compared with the original’s and agrees.

Jargon

AsmEvo: The system under discussion: it takes a compiled AMD GPU kernel, has an agent propose assembly-level edits, and accepts an edit only if a verification step shows the rebuilt kernel still behaves like the original. It optimizes an existing binary rather than writing a kernel from scratch.
AMDGPU code object: The compiled artifact AMD GPUs actually load and run: machine instructions plus the metadata the runtime needs to launch them. It is the deployed form of a kernel, the end product of whatever Triton, HIP or CUDA-style source produced it.
Reassemblable representation: A version of that binary you can edit as text and turn back into a working object. Recovering one is the enabling step: without it there is nothing to hand an agent, because a raw binary is not something you can meaningfully rewrite line by line.
ABI: Application Binary Interface — the contract about how a compiled function is called: which registers hold which arguments, how the launch is set up, what the runtime expects to find. An edit that breaks it produces a kernel that no longer fits its caller, however fast it runs.
Differential verification: Running the edited kernel and the original one on identical launches and comparing the results. It is the accept/reject gate here: the paper treats the deployed binary as the reference behaviour, so a candidate is judged against the original's observed behaviour rather than against a specification.
Hot window: The small region of the kernel where the time is actually spent, identified by profiling. Editing is aimed there rather than across the whole binary, which keeps the search small.
KernelBench: A benchmark suite of GPU kernels used here to measure the approach; AsmEvo reports results on a selected subset of it, so the headline numbers describe those kernels rather than kernels in general. A neighbouring idea — grading LLM-written kernels by the GPU time they really burn — is covered in Atrex-Bench.
AITer: AMD's own library of inference kernels, and one of the three binary families measured here. It is the family on which AsmEvo reports its smallest gains.

The news. On August 21, 2026, a paper introduced AsmEvo, which studies kernel optimization in the setting where the deployed AMDGPU binary — not the source — is the thing you have to work with. It reconstructs a reassemblable representation of the code object, has a long-horizon agent propose low-level edits, rebuilds an ABI-preserving object, and accepts a candidate only after differential verification against the original under identical launches. The reported results cover a selected KernelBench subset on MI308X plus AMD's AITer binaries and vLLM/SGLang Triton assembly kernels on MI300X. Read the paper →

Picture the tuner again, standing in front of an engine that is bolted shut. There are no blueprints, and the drawings that do exist describe something several revisions upstream of the metal in front of them. In that situation the engine itself stops being a copy of the design and becomes the design — the only object whose behaviour counts, and therefore the only object worth editing. That is the premise AsmEvo starts from, and the paper states it plainly: "the deployed binary is the only behavioral oracle."

Most of the tooling a GPU programmer reaches for sits above that level. The abstraction ladder runs from eager PyTorch down through torch.compile, then Triton, then CUDA C++ — each rung trading away convenience for control, and each one still trusting a compiler to translate your intent into machine code faithfully.

GPU Abstraction Stack

PyTorch EagerL4

Write Python ops — runs immediately, one kernel per op

torch.compileL3

Annotate with @torch.compile — compiler fuses & optimizes for you

TritonL2

Write tile-level kernels in Python — full control, no C++ needed

CUDA C++L1

Write thread-level kernels — maximum control, maximum complexity

↑ easier↓ more control

AsmEvo works one rung below the bottom of that ladder. The trade it makes is the same one the ladder has been making all along, just pushed one step further: give up the readable source, get the last slice of control. That is exactly the choice the curriculum frames as the CUDA expertise gap and as picking which rung to work on — AsmEvo simply adds a rung underneath and asks whether an agent can stand on it.

The obvious objection is that editing assembly by hand-waving is how you get a fast kernel that quietly computes the wrong answer. This is where the bench test earns its place. AsmEvo rebuilds each candidate as an ABI-preserving object so the caller still fits, then runs differential verification: edited kernel and original, identical launches, results compared. Only a candidate that survives that comparison is timed at all. The stopwatch does not start until that comparison agrees — which turns the whole thing into an evaluator-optimizer loop whose evaluator is not a score at all, but a pass/fail gate. Speed is a ranking; correctness is admission. And because a whole binary is far too large a search space, profiling narrows the edits to the hot window — the region that is actually limiting the kernel — with a conservative in-place patch available as a fallback.

What the measurements show is a wide spread, and the spread tracks which family of binaries was being optimized.

Binary family	Hardware	Geometric-mean speedup	Max speedup	Coverage
Selected KernelBench kernels	MI308X	1.35×	3.88×	29 of 30 improved
AITer binaries (AMD)	MI300X	1.09×	1.31×	reported, count not stated
vLLM / SGLang Triton assembly kernels	MI300X	1.18×	1.34×	reported, count not stated

Put those ratios into the form you actually care about — the share of a kernel’s time that disappears — and the spread becomes readable, because the arithmetic is just 1 − 1/speedup. A 1.35× geometric mean (the average of the per-kernel speedups, computed multiplicatively) removes about 26% of the time; the 3.88× best case removes about 74%. Now hold the hardware fixed and compare only the two MI300X families, which is the one comparison the reported numbers license: 1.18× on the vLLM/SGLang Triton assembly kernels removes about 15%, while 1.09× on the AITer binaries removes about 8% — roughly half as much on AMD’s own inference-kernel library. Made concrete on a kernel that costs 4.00 ms per launch (illustrative — the paper reports ratios, not absolute times), that is 4.00 / 1.18 = 3.39 ms, a saving of 0.61 ms, against 4.00 / 1.09 = 3.67 ms, a saving of 0.33 ms. The paper does not claim a cause for that gap, and neither should you — but the reading worth keeping is that a speedup number here says as much about which binary you pointed the optimizer at as it says about the optimizer.

Which is what makes this worth reading even if you will never touch an AMD code object. The transferable part is the shape of the loop, not the target: when an agent is turned loose on something where a plausible-looking change can be silently wrong, the useful design move is to make correctness the admission gate and performance only the ranking among what is already admitted. A kernel binary is an unusually clean case of that, because agreement can be checked by execution rather than argued about — but the same shape carries over to other optimizers whose search space contains a great many fast, wrong answers.

Goes deeper in: GPU & CUDA → Triton & torch.compile → When to Use What

Related explainers

Atrex-Bench — Trace-weighted kernel benchmarking — the measurement side of the same problem: which kernels and shapes deserve the optimization effort in the first place
Fused INT8 GEMM kernel — the same last-slice-of-performance hunt one rung higher, written by hand in Triton rather than searched for in assembly
The Verification Horizon — Co-evolving verifiers — what happens when the gate is a learned judge instead of an execution check, and the agent starts optimizing against the gate
Dockerless — Execution-free patch verification — the opposite bet: judging correctness by reasoning instead of by running the thing

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based