What is analytical kernel selection?

It chooses a GPU kernel configuration by computing predicted latency from a performance model of the hardware, instead of running and timing candidates. In Meta and AMD's paper, the model is called Origami and it selects GEMM tile shapes.

How is it different from autotuning?

Autotuning measures candidates on the real GPU and stores the best per shape, so each new shape needs a run or a table entry. An analytical model evaluates a formula for each candidate, so a new shape costs microseconds and needs no warmup.

How accurate is Origami?

The paper reports 93% average selection efficiency compared with exhaustively autotuned GEMM performance across datatypes and transpose configurations. Efficiency here means how close the chosen kernel gets to the best one found by exhaustive search.

Does this only work on AMD GPUs?

The model is calibrated for the MI350X, but the authors argue the formulation is general because GEMM has regular tiling and predictable data movement. They say retargeting is largely a matter of substituting hardware constants such as cache sizes and achievable bandwidth.

GPU· 2026-10-19

Meta on AMD MI350X — Origami analytical kernel selection vs autotuning — What does it mean?

Jargon

GEMM: General matrix multiply, C = A × B. Neural-network layers are built from GEMMs, so one kernel choice for a shape is reused every time that shape appears.
Tile: A small block of the output matrix that one workgroup computes. The tile's size decides how much data is reused and how many workgroups exist. See Why Tiling Works.
Autotuning: Running many kernel configurations on the real hardware, timing each, and keeping the fastest. It is accurate but slow, and the result is valid only for the shapes you tried.
Compute unit (CU): AMD's name for the block of the GPU that runs workgroups, roughly what NVIDIA calls a streaming multiprocessor. The MI350X has 256 of them.
LDS: Local data share, the fast on-chip scratchpad inside a CU. AMD's counterpart of CUDA shared memory; tiles are staged here.
Stream-K: A way to split a GEMM along the K dimension across CUs, so that work divides evenly even when the number of output tiles does not.
Wave quantization: The waste that appears when the number of tiles is not a multiple of the number of CUs, so the last round of work leaves some CUs idle.
Roofline model: A performance bound set by either peak compute or memory bandwidth, whichever binds first. See Reading the Roofline Plot.

The news. On October 15, 2026, Meta and AMD posted Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs (arXiv 2610.22230), a case study of training recommendation models on AMD Instinct MI350X GPUs instead of NVIDIA B200. It reports up to 1.5× end-to-end training throughput over a directly ported baseline, and says the changes were upstreamed to PyTorch, Triton and ROCm. One piece is an analytical model, Origami, that chooses GEMM kernels with no tuning run. Read the paper →

Measure the truck, don't reload it

Suppose you must pack a moving truck for thousands of different moves. One approach is to test-load the truck with every layout, time each attempt, and write the winner in a notebook. It works, but only for the moves already in the notebook, and a different truck means starting the notebook over. The other approach is to measure the truck once, then work out on paper how long each layout would take to load and to drive, and pick the best answer in seconds.

That is the difference between autotuning in a compiler like Triton and Origami. The paper says earlier hipBLASLt selection was offline autotuning: enumerate kernel configurations on representative shapes and store the best in lookup tables. That fails at scale, because recommendation models execute thousands of unique GEMM shapes per model and an exhaustive table for every shape is impractical.

Origami replaces the sweep with a model of the GEMM's critical path through the memory and compute hierarchy. For each candidate tile shape it estimates two latencies and keeps the shape with the lowest total. Each candidate evaluates in nanoseconds, so the full search takes microseconds.

What the paper's arithmetic looks like

The model splits the output matrix C into tiles, one per workgroup. A workgroup walks along the K dimension, loading a tile of A and a tile of B, multiplying, and writing back. Compute latency is the number of matrix instructions per tile times each instruction's latency. Memory latency is the largest of the times to move the traffic through L2, the last-level cache (LLC) and HBM, where each level only sees the bytes that missed the level above it.

The cache hit rate itself is predicted geometrically. If M_C × N_C workgroups share a cache, A tiles are reused across N_C workgroups and B tiles across M_C, giving the paper's hit rate H = 1 − (M_C·M_T + N_C·N_T) / (M_C·N_C·(M_T + N_T)). The loop overlaps loading with computing, so a tile takes the prologue load, plus one step per K-slice at the larger of compute and memory time, plus the final store. That max is the roofline idea applied to one tile: the slower side sets the pace.

Two more effects are modelled. An issue-rate bound limits throughput when too few vector lanes are issuing loads, which pushes toward smaller tiles while reuse pushes toward larger ones. And the model counts wave quantization, then chooses tile shapes and a Stream-K grid size to shrink the idle tail.

Question	Offline autotuning	Origami (analytical model)
How is a kernel chosen?	Run candidates on the GPU, keep the fastest	Compute each candidate's latency from hardware constants, keep the lowest
Cost per new matrix shape	A tuning run per shape, or a stored table entry	Microseconds for the full search, no warmup
Moving to a new chip	Re-tune and rebuild the tables	Largely substituting hardware constants such as cache sizes and achievable bandwidth, per the paper
Accuracy	Best observed by construction, for tuned shapes	93% average selection efficiency versus exhaustively autotuned GEMM, per the paper

A worked example

Two small calculations show why the model needs no stopwatch. First, wave quantization, using the paper's own picture: an output of 4 × 5 = 20 tiles on 8 CUs takes 3 rounds, because 20 / 8 rounds up to 3. The last round has 20 mod 8 = 4 tiles, so 4 of 8 CUs sit idle and utilisation is 20 / 24 ≈ 83%. A cost model sees this from the shape alone; choosing a tile size that makes 24 tiles would remove it.

Second, the hit-rate formula with illustrative numbers: 128 × 128 tiles (illustrative) and a 4 × 4 grid of workgroups sharing one cache gives H = 1 − (4·128 + 4·128) / (16·256) = 1 − 1024 / 4096 = 0.75. Sharing across an 8 × 8 grid gives 1 − 2048 / 16384 = 0.875. More workgroups sharing the cache means less traffic reaches HBM, and that lowers the memory term in the max. Neither number required running a kernel.

Where it earns its keep, and where it stops

The model works because GEMM is regular: uniform tile shapes and predictable data movement, so, in the authors' words, its cost reduces to a hardware-parameterized roofline. It is tuned for MI350X, and the paper reports it is also integrated into tritonBLAS for zero-warmup selection at runtime. Irregular kernels do not have this property, so the paper still tunes Triton, CUDA and Inductor kernels by hand and with agentic workflows. The 1.5× end-to-end figure is the combined result of model changes, kernels and software, not of Origami alone, and the paper does not split that gain by component in the sources summarised here.

Goes deeper in: GPU & CUDA → Triton & torch.compile → How Triton Works

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based