What is a hierarchical execution reward?

A reward for generated code built from several graded terms instead of one pass/fail bit. In AMDKernelVault it adds **0.4 for compiling, 1.0 for matching the reference output and up to 1.5 for a capped speedup**, giving a total in the range 0 to 2.9.

Why do binary rewards stall GRPO training?

GRPO uses a group's own average as the baseline, so a candidate only learns from how it differs from its peers. With a 0/1 reward and a low success rate, whole groups come back all-zero, every advantage is zero, and those groups contribute nothing to the gradient.

How is this different from having a model judge the kernel?

The score here is produced by the toolchain, not by an opinion. A compiler either builds the kernel or does not, the output either matches the PyTorch reference or does not, and the profiler reports a measured latency — which is why the corpus is described as execution-verified.

Does the 8B model beat frontier models at writing kernels?

On correctness within a fixed attempt budget, the paper reports it ahead of the models it compares against — 41.94% Corr@3 on ROCmBench versus 35.48% for Claude Sonnet 4. It does not uniformly lead on compilation rate or speed, and the benchmarks cover a few hundred kernels rather than every AMD workload.

GPU· 2026-09-22

AMDKernelVault trains an 8B model to write AMD GPU kernels — Hierarchical execution reward — What does it mean?

TL;DR

What is it: AMDKernelVault is an open HIP and Triton kernel corpus plus a training recipe, published by AMD on September 11, 2026 — and the durable idea inside it is its hierarchical execution reward, a kernel-generation score assembled from compiling, passing and profiling rather than from a single pass/fail bit.
Why it’s needed: Reinforcement learning over generated GPU kernels sits in a low-success regime: a small model almost never writes a correct kernel on the first try, so a pass/fail reward hands the same zero to every candidate in a group and the update carries no information about which attempt came closest.
vs previous: Standard RLVR grades a rollout with one verifiable bit — correct or not. AMDKernelVault splits that bit into three terms that add up — 0.4 for compiling, 1.0 for correctness and up to 1.5 for a capped speedup — so near misses separate from total failures, while a lucky timing measurement still cannot outrank being right.

Jargon

GRPO: Group Relative Policy Optimization. The model generates a group of candidates for one prompt, and each candidate's advantage is its reward minus that group's own average — there is no separate value network to supply a baseline.
Advantage: How much better one candidate was than its group's average. If every candidate in a group scores the same, every advantage is zero and that group contributes nothing to the update.
RLVR: Reinforcement Learning with Verifiable Rewards. The reward comes from running a checker rather than from a learned preference model. Full explainer
HIP: AMD's C++ GPU programming interface — the ROCm stack's counterpart to CUDA C++, with a deliberately similar surface so CUDA code can be ported.
ROCm: AMD's GPU software stack: compiler, runtime and libraries. A generated kernel has to compile and run under it to count as verified here.
Triton: A Python-embedded language for writing GPU kernels at the block level, compiled down for you instead of hand-written per thread. Track step
SFT cold start: Supervised fine-tuning run before the RL stage, so the policy already succeeds often enough that RL has successes to learn from at all.
Corr@3: The share of tasks where at least one of three attempts produced a correct kernel. Pass@1 is the same idea with a single attempt.

The news. On September 11, 2026, AMD released AMDKernelVault, an open corpus and training framework for generating GPU kernels on its recent CDNA data-center GPUs. Two agent pipelines — HIPKernelGen and TritonKernelGen — turn PyTorch reference implementations into HIP or Triton kernels, compile them under ROCm, and latency-profile them on AMD hardware; what survives becomes training data. The corpus holds 62,153 execution-verified HIP samples, 2,377 ROCm library question-answer entries and 39,893 Triton kernels. AMD then fine-tuned Qwen3-8B on it and ran execution-aware reinforcement learning on top. Read the paper

Keep the two boards in view. A coach who records only bullseyes is running the scheme most verifiable-reward training uses: a rollout is correct or it is not, and the score is 1 or 0. GRPO then compares each shot to the round's own average, and that comparison — the advantage — is the only thing the update reacts to. When every arrow in the round misses, every score is 0, the average is 0, and every advantage is 0, so the round teaches nothing at all.

Kernel writing sits squarely in that regime. Getting a correct HIP or Triton kernel out of an 8B model is rare enough that most rounds come back empty, and an empty round is a silent one. Painting rings on the board is the fix. AMDKernelVault replaces the lone bullseye with three bands that add up: 0.4 once a kernel compiles and matches the expected interface, 1.0 once its output matches the PyTorch reference, and up to 1.5 more for a measured speedup — a total somewhere between 0 and 2.9. A kernel that merely builds now outranks one that will not parse, and the round finally has something to say. The speed term is capped on purpose: without a ceiling, one freak timing measurement would outweigh a kernel that is simply right.

PyTorch referencethe task

Model writes a kernelHIP or Triton

Compile under ROCm0.4 if it builds

Test vs reference1.0 if correct

Profile on deviceup to 1.5 for speed

GRPO ranks the groupa gradient, not a tie

Reward term	Value	What earns it	Source
Compilation	0.4	The kernel builds under ROCm and matches the expected interface	Table 8
Correctness	1.0	Output is functionally equivalent to the PyTorch reference	Table 8
Speedup	up to 1.5	A bounded term for measured latency gains, capped so timing outliers cannot dominate	Table 8

Two more pieces keep the signal alive, and they are worth separating from the reward itself. On the Triton side the loop is multi-turn: the model writes a kernel, reads the compiler or runtime error, reflects on it, and writes again — three turns per task. Score a single attempt instead and correctness on TritonBench-G falls from 33.2% to 20.1%. The RL stage also does not begin from the base model: supervised fine-tuning runs first, which the paper reports lifts the base model's success rate from roughly 15% to roughly 51%, and skipping it drops the final result to 8.5%. The graded reward, the retry loop and the cold start are three attacks on one problem — making sure a training round contains a spread of outcomes rather than a wall of zeros.

Work the odds once and the reward shape stops looking like a tuning detail. Hold two numbers fixed, both taken from the paper's own corollary: a per-sample success probability of 6%, and a GRPO group of 8 candidates. Under a binary reward a round is wasted whenever all 8 land on the same score — all 8 fail, or all 8 pass. All-pass is negligible at this success rate: 0.06 raised to the 8th is under one in a billion. All-fail is not: 0.94 raised to the 8th is about 0.61. Add the two and about 60% of training rounds hand back zero advantage to every candidate in them — three rounds in five contributing nothing (the paper states the figure as 60.6%). The graded reward attacks that number head-on: the authors measure partial compilation credit raising the share of samples carrying usable gradient from near zero to approximately 44%.

The honest reading of the results is narrower than the headline. Under a fixed three-iteration budget, the trained Qwen3-8B leads the compared models on correctness — 34.0% Pass@1 on PyTorch-to-HIP, 33.2% Corr@3 on TritonBench-G and 41.94% Corr@3 on ROCmBench, against 29.8% and 35.48% for Claude Sonnet 4 on the latter two. It does not uniformly lead on compilation rate or on speed, which the abstract says outright. The evaluation covers 200 PyTorch-to-HIP tasks, 100 HIP-to-HIP optimization tasks, 184 TritonBench-G kernels and 31 ROCmBench kernels, and the authors note these sets do not establish performance across every AMD operator or workload family. The paper also comes from AMD, whose hardware it targets. What it demonstrates convincingly is the training signal — not a finished kernel engineer.

Goes deeper in: AI Agents → Evals & Diagnostics → Pass/Fail vs Score

Related explainers

AsmEvo tunes AMD GPU kernels at the assembly level — the same hardware, the opposite end of the stack: rewriting a compiled kernel's assembly instead of training a model to write one.
VPO — vector-reward advantage vs GRPO scalar collapse — a second way GRPO collapses, from squashing several objectives into one scalar rather than from a reward with too few levels.
SIS turns off-policy RL tokens into on-policy updates — what to do once the rounds do carry signal but the rollouts that produced them are stale.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based