E8-QAT is a 2-bit quantization-aware training method. It treats each group of eight weights as a point in eight dimensions and snaps it to the nearest point of the E8 lattice using rounding and a parity check, so it needs no stored or searched codebook during training.

Why use a lattice instead of a learned codebook?

A learned or explicit codebook must be stored and searched at every training step, and its size grows exponentially with group length at a fixed bit budget. A lattice is a formula, so the nearest point costs about as many steps as there are weights in the group.

What is an escape record in E8-QAT?

Training can move a group to a lattice point outside the finite 65,536-point core. Instead of clipping it, E8-QAT stores a 20-byte record with its exact coordinates. In the paper's 750-step run these records added only 0.086% to the 2-bit index payload.

Does E8-QAT also compress activations or the KV cache?

No. The paper quantizes weights only. Activations and the KV cache stay in higher precision, and the authors list combining the lattice weights with KV-cache quantization as a direction for further memory savings.

LLM· 2026-10-22

E8-QAT 2-bit training — E8 lattice implicit codebook — What does it mean?

TL;DR

What is it: E8-QAT is a 2-bit quantization-aware training method, described in a paper submitted on October 20, 2026, that snaps every group of eight weights to a point on the E8 lattice by arithmetic instead of looking it up in a stored codebook.
Why it’s needed: Vector quantization stores weights more accurately than rounding each weight alone, but a lookup table searched at every training step is too slow; a formula that finds the nearest codeword in about eight steps removes that search cost, and the paper reports 80% lower peak memory and a 4× larger batch on Qwen3.6-35B-A3B.
vs previous: The previous approach, explicit-codebook vector quantization, must store its codewords, and a straightforward search scans all 65,536 of them per group at 2 bits and 8 weights; the lattice replaces that search with rounding plus a parity check.

Jargon

QAT: Quantization-aware training. The model keeps training while its weights are forced onto the low-bit grid in every forward pass, so it learns to work around the rounding error.
PTQ: Post-training quantization. Weights are compressed once after training with no further learning, which is cheap but loses accuracy at very low bit widths.
Vector quantization: Replacing a whole group of weights with the nearest entry of a codebook, instead of rounding each weight alone. Related: trellis-coded weight quantization.
E8 lattice: A classical eight-dimensional grid from sphere-packing mathematics. Its points have all-integer or all-half-integer coordinates whose sum is even. It is a fixed formula, not something learned from the model.
Straight-through estimator (STE): A training trick. Rounding has zero gradient almost everywhere, so the backward pass pretends the rounding step was the identity and passes the gradient through unchanged.
Hadamard preprocessing: A random sign flip followed by a Hadamard rotation that mixes the weights in a block. The mixed values look roughly Gaussian, which suits a ball-shaped set of codewords.

The news. On October 20, 2026, a team from Peking University, City University of Hong Kong and Tencent submitted E8-QAT, a 2-bit quantization-aware training framework. On Qwen3.6-35B-A3B, the authors report that it matches the full-precision average across four tasks, cuts peak memory by 80% and supports a 4× larger batch. Read the paper →

Picture a warehouse that has to shelve one parcel after another, each described by eight measurements. The old way is to carry a printed catalog of every legal shelf and walk it page by page until you find the closest match. At 2 bits per weight and eight weights per group, that catalog has 65,536 entries, and a training run needs a lookup for every group of weights in the model, on every forward pass.

E8-QAT builds the warehouse differently. The shelves sit on a grid laid out by a rule, so you never open a catalog: you compute the shelf from the parcel's measurements. That grid is the E8 lattice, and the computation is rounding. Because the shelf positions are a formula, no codebook is stored and none is searched during the training forward pass.

The lattice is a fixed mathematical object: points whose eight coordinates are all integers or all half-integers, with an even sum. Finding the nearest lattice point to a weight group takes three moves, which the paper attributes to Conway and Sloane (1982): round the coordinates, repair the parity, and compare a constant number of candidates. In the classical decoder, the repair moves the single worst-rounded coordinate to its other neighbouring value whenever the rounded sum is odd. That is a handful of operations per group, about the same as the eight weights themselves, instead of a scan over 65,536 codewords.

Two pieces of surrounding machinery make the formula work on real weights. First, weights are mixed by a Hadamard transform so they look roughly Gaussian (the authors measured kurtosis of about 3.02 against the Gaussian value of 3), which matches a ball-shaped set of lattice points. Second, training uses an identity straight-through estimator, so the hard snap in the forward pass does not block gradients. Compare this with the quantization process you may know from scalar rounding: the snap is the same idea, but the grid is eight-dimensional.

Approach	Snaps	Search per group (8 weights, 2 bits)	Stored codebook
Scalar rounding	each weight alone	none: one rounding per weight	none
Explicit-codebook VQ	a group to its nearest codeword	O(dK) = 8 × 65,536 = 524,288 operations (computed from the paper's cost formula)	65,536 codewords
E8 lattice, as in E8-QAT	a group to its nearest lattice point	O(d), about 8 steps, independent of codebook size	none during training; a shared decoding table is optional at inference

Training happily wanders, though, and the lattice has infinitely many points. For compact storage the paper keeps a finite core: the first 65,536 lattice points ordered by squared norm, which is 56,881 points up to squared norm 10 plus 8,655 points from the shell at squared norm 12. Each group stores a 16-bit index into that core, which is exactly 2 bits per weight for eight weights.

Sometimes training pushes a group to a lattice point outside the core, which the paper calls an escape. Clipping it back would change the weights the model was trained with, so each escape instead gets a 20-byte record holding its position and exact coordinates. The payload is then 2 + 20ρ bits per weight, where ρ is the fraction of escaped groups. This is the overflow shelf with a handwritten slip.

Here is the cost in numbers, with a made-up but typical layer (illustrative). Take a 4,096 × 4,096 weight matrix: 16,777,216 weights, or 2,097,152 groups of eight. Exhaustive search at 524,288 operations per group would be about 1.1 × 10^12 operations per forward pass for this one layer, while the lattice projection needs on the order of 8 per group, roughly 65,536 times fewer. Now the storage side. The paper's measured escape fraction after 750 training steps was ρ = 8.556 × 10^-5, so about 179 of the 2,097,152 groups escape. At 20 bytes each that is about 3.6 KB of records on top of a 4.2 MB index. That is the paper's 0.086% overhead, so the model still ships at essentially 2 bits per weight.

What the authors report on Qwen3.6-35B-A3B at 2 bits: E8-QAT beats the strongest vector-quantization QAT baseline they compare by 2.03% on IFEval and 1.51% on LiveCodeBench under the same data, schedule and bit width. Peak memory falls 80% against BF16, the batch can be 4× larger, and at each format's largest feasible batch, decode throughput reaches 1.97× BF16. In the memory-limits step of batching, the reason is the same: smaller weights leave room for more concurrent sequences. Before QAT starts, the E8-based post-training starting point had the lowest layer-wise distortion in 39 of 40 layers.

The limits matter. These are the authors' numbers on one mixture-of-experts model plus Llama-3.1-8B, with long-form and agentic evaluations focused on the MoE model. Only weights are quantized; activations and the KV cache stay in higher precision. And the 1.97× compares two different batch sizes, not the same batch. For why 2-bit weights lose accuracy without care, see the outlier problem.

Goes deeper in: LLM Internals → Quantization → Modern Methods

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based