Decompose W4A4 quantization error into correctable components — Activation-guided weight compensation vs orthogonal residual — What does it mean?

The news. On September 18, 2026, Yamato Narita and Issei Sato of the University of Tokyo posted Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals. Rather than proposing another 4-bit recipe, it derives an exact decomposition of the error a W4A4 linear layer makes and uses the part weight optimization cannot reach to explain why the existing transforms work. A configuration assembled from those guidelines, L2-SmoothRot, is reported competitive with gradient-trained SpinQuant across eight Llama and Mistral models, without any backpropagation. Read the paper →

Picture a wet window and a squeegee. You draw the blade down and a clean band appears behind it. You press harder, draw again, press harder still — and a set of streaks running across your stroke stays exactly as it was. No amount of force on that blade, at that angle, reaches them. To clear them you have to turn the blade.

That is the shape of the paper's Theorem 1. A quantized linear layer makes an error, and the authors show it splits cleanly in two. The first piece is the activation-guided weight compensation term: the part of the error that lives in the space the quantized weights can actually reach, so a solver can nudge the weights to absorb much of it — though the weights are themselves quantized to four bits, so this is a reduction, not a clean cancellation. The second piece lives in the orthogonal complement of that space — geometrically at right angles to everything the weights can express. For a fixed transformation and fixed inputs, weight optimization can shrink the compensation term but cannot move the orthogonal residual at all.

This reframes what the pieces of a 4-bit pipeline are for. Weight quantizers in the GPTQ family are stroke pressure: better solvers, better calibration, error compensation across columns — all of it works on the first term. Rotation and channel scaling are the turn of the blade: they change the geometry itself, which is the lever this analysis leaves for the second term. The asymmetry is the point: for fixed inputs and a fixed transformation, the weight solver is confined to the first term, while changing the transformation moves the geometry underneath both. So a second weight solver is not a substitute for a transform, and adding a transform changes the job the weight solver then has to do.

Four-bit integers give you sixteen levels, and those levels have to stretch across whatever range the tensor contains. That is survivable for weights, whose values are known and well-behaved. Activations are the problem: a handful of hidden-dimension channels stay large on almost every token — persistent channel-wise outliers — and a per-token scale sized to hold them leaves ordinary values rounding into a few coarse buckets near zero. This is the long-standing outlier problem that SmoothQuant, QuaRot and SpinQuant were all built to attack.

The paper's bound on the residual is what makes those attacks legible. It splits into two quantities: one driven by the persistent outlier channels, one by the ordinary bulk. Rotation goes after the first. A Hadamard matrix has every entry at magnitude one over the square root of the width, so after rotation each outlier channel contributes a small amount to every coordinate instead of a huge amount to one. But the contributions still have signs. Under a fixed Hadamard rotation those contributions can line up and reinforce each other, so the bound grows with the square of the outlier-channel count; giving the rotation random signs breaks the alignment and the bound grows only linearly.

That is a sharper statement than "rotation spreads outliers out," and it has a practical edge. The authors note that in QuaRot, random signs are used only on the offline rotation while the online Hadamard rotations stay fixed — so the constructive-interference case is exactly the one left in place at the attention output and the feed-forward down projection. Their signed online rotation puts random signs there too, and applies the same signed rotation to queries and keys after the usual rotary position embedding (RoPE), which preserves the unquantized attention scores because the two rotations cancel in the product before the keys are quantized.

Worked example — what random signs actually buy you. This compares the paper's two upper bounds, not measured errors: a smaller bound does not by itself guarantee a smaller real error. Hold three things fixed — the number of tokens, the largest scaled outlier magnitude, and the layer itself — and the two bounds on the outlier term differ only in how they depend on the number of persistently large channels. Take a hidden width of d = 4096, a failure probability of delta = 0.05, and N = 64 persistent outlier channels. All three of those values are illustrative; the paper states the bounds, not these settings.

Under a fixed rotation the bound carries a factor N^2 / d, which is 64^2 / 4096 = 1.00. Under random signs it carries 2N(log 2d + log 1/delta) / d. With log(2 x 4096) = 9.01 and log(1/0.05) = 3.00, that is 2 x 64 x 12.01 / 4096 = 0.375. So on these numbers the random-sign bound lands at about 0.375x the fixed-rotation bound — roughly 2.7x smaller — for a change that adds only a sign flip per element.

Run the comparison backwards and it marks where the randomized bound stops being the tighter of the two — which is a statement about the bounds, not a threshold below which random signs stop helping. The two expressions cross at N = 2(log 2d + log 1/delta), which at this width is about 24 channels. Below that, the quadratic factor is the smaller of the two, so the bound itself no longer argues for random signs. That matches the paper's own framing: the outlier-channel count and their scaled magnitude both vary by model and by layer, which it offers as one explanation for why different models degrade so differently under low-bit quantization.

The second quantity in the bound — the ordinary bulk, once the outliers have been spread — is what channel scaling is for, and this is where the paper's tidiest result sits. Bounding that term by its second moment and taking the tractable per-channel surrogate gives a scaling rule: set each channel's constant to the square root of the ratio of the activation column's L2 norm to the weight column's L2 norm. Relax that bound twice — replace the weight matrix's worst-case amplification (its spectral norm) with its total energy (its Frobenius norm), then replace per-channel second moments with per-channel maxima — and the same derivation hands you the square root of the ratio of the two maxima instead. That second expression is SmoothQuant's familiar rule in its balanced, square-root form: the same bound, relaxed twice.

So the framework's claim is not that SmoothQuant is wrong. It is that this square-root setting of SmoothQuant sits one relaxation further out than it needs to once you have already applied a random-sign rotation, because the rotation is what makes second-moment statistics the right summary of the bulk in the first place. The L2 rule keeps the tighter statistic; the L-infinity rule throws it away for channel maxima.

Assembled, those three choices — signed online rotation, calibration-based sign selection, and L2 channel scaling — make up L2-SmoothRot. Its selection step is deliberately cheap: screen ten random sign patterns with round-to-nearest quantization, re-check the best three with the full GPTAQ solver, and keep the lowest validation perplexity. The reported result across Llama-7B/13B, Llama2-7B/13B, Llama3-8B, Llama3.2-1B/3B and Mistral-7B-v0.3 is the lowest WikiText-2 and C4 perplexity on all eight models, and the highest average zero-shot accuracy on five of them. The whole configuration is chosen without a single backward pass, and still lands within 0.26 percentage points of gradient-trained SpinQuant's average accuracy on Llama3-8B.

The honest limits are worth naming. The decomposition is local — it is stated for one linear layer against that layer's own full-precision output, not for the network end to end, so it says nothing directly about how error compounds through depth. The outlier and bulk quantities appear only in an upper bound on the residual, not in the exact decomposition, so they guide design rather than measure it. And the accuracy picture is genuinely mixed: on Llama3.2-3B the method improves perplexity over SpinQuant while trailing its average accuracy by 1.09 points. What it buys is not a new ceiling. It is an account of why the existing tricks work, which is the thing you need when a model quantizes badly and you have to decide whether to reach for a better solver or a different transform.

Goes deeper in: LLM Internals → Quantization → The Outlier Problem

Related explainers

• Why post-training quantization works — that explainer asks a different question about the same failure surface: why 4-bit rounding costs so little accuracy across a whole network, rather than how one layer's error splits.
• KVarN's Hadamard rotation for a 2-bit KV cache — the same rotation trick, pointed at the KV cache instead of the weight-activation product. Useful for seeing where else the outlier argument lands.
• OScaR and token norm imbalance — that explainer argues the outliers that survive rotation in an INT2 KV cache sit on the sequence axis rather than the channel axis. Different setting, but a useful counterweight to the channel-wise framing used here.