**Losslessly Compressed Tensors** is a GPU framework that stores BF16 weights, activations and optimizer states in compressed form during training. It Huffman-codes only the exponent byte and leaves the sign-fraction byte alone, so every value decodes exactly.

Why compress only the exponent?

In the paper's modelled distributions the exponent of a trained tensor carries only about 3 bits of information in an 8-bit field, while the sign and fraction bits are much less predictable. LCT codes the exponent to capture the main redundancy and leaves the sign-fraction byte uncompressed, even where it may hold some.

How is this different from quantization?

Quantization such as FP8 or 8-bit optimizer states **rounds values to fewer bits**, which can change results and needs tuning. LCT is lossless, so the training run behaves as the uncompressed run would, at the price of about 7 to 8% longer steps in the reported experiments.

Does it really give 3.5 times longer context?

In the reported Qwen3-4B fine-tuning at batch size 1, the baseline ran out of memory above 1000 tokens and LCT reached 3500 tokens, so the 3.5× figure compares the longest tested lengths. It is an experimental result from a preprint under review, not a guaranteed ceiling.

GPU· 2026-10-17

LCT paper — Lossless Huffman coding of BF16 training tensors — What does it mean?

TL;DR

What is it: LCT (Losslessly Compressed Tensors) is an October 2026 preprint that stores BF16 training tensors as an untouched sign-fraction byte plus a Huffman-coded exponent, so every value comes back bit-for-bit identical.
Why it’s needed: Training memory, not compute, often decides the model size and context length that fit on one GPU, and the exponent byte of a trained tensor is repetitive (about 3 bits of real information under the paper's modelled distributions), which let the reported runs reclaim roughly a quarter of peak memory without changing any number.
vs previous: Lossy methods such as FP8 training, activation quantization and 8-bit optimizers trade accuracy for bytes and need tuning; LCT keeps every value exact, and its GPU codec is built for the repeated encode-decode loop that one-shot inference compressors were not designed for.

Jargon

BF16: A 16-bit floating-point format with 1 sign bit, 8 exponent bits and 7 fraction bits. It keeps the same range as 32-bit floats but with less precision. See GPU & CUDA → Tensor Cores → Precision Formats.
Exponent: The part of a floating-point number that sets its magnitude (roughly, which power of two it is near). Values in a trained network cluster around a few magnitudes, so exponents repeat a lot.
Huffman coding: A lossless scheme that gives short bit patterns to common symbols and long ones to rare symbols, so the average length drops toward the symbol entropy.
Entropy: The minimum average number of bits per symbol that any lossless code needs, given how often each symbol occurs. A skewed distribution has low entropy.
Optimizer state: Extra tensors the optimizer keeps for every weight, such as the momentum buffer in SGD with momentum. They are the same size as the weights, so they double the memory of a naive training setup.
Fused kernel: One GPU kernel that does several steps in a single pass so intermediate results never travel to HBM and back. See GPU & CUDA → Operator Fusion.

The news. On October 15, 2026, researchers from the University of Cambridge and the University of Oxford posted LCT, a GPU framework that losslessly compresses weights, activations and optimizer states during training. It is a preprint under review. On a 162M-parameter NanoGPT it cut peak memory by 24.6% for a 7.2% longer step, and in Qwen3-4B fine-tuning at batch size 1 it let sequences up to 3.5 times longer than the baseline's limit fit on the same GPU. Read the paper →

Think of every BF16 number as a bag with a luggage tag that has two fields: the contents (the sign and fraction bits) and the destination (the exponent). The contents are much less predictable, so LCT leaves that byte uncompressed and copies it across as it is. The destination is a different story. In a trained network most values are small and sit near a handful of magnitudes, so a few destinations account for nearly every bag. LCT gives those popular destinations a short code and stamps the rare ones OTHER, followed by the full address, which means nothing is ever lost.

In bits, a BF16 value is 1 sign + 8 exponent + 7 fraction = 16 bits. LCT splits it into two bytes: a sign-fraction byte that stays uncompressed and an exponent byte that is entropy-coded. For the Gaussian, Laplace and power-law models the paper uses, the ideal cost of an exponent is only about 2.5 to 3.1 bits when the model matches the data. That gap between 8 stored bits and about 3 informative bits is the whole opportunity. LCT uses Huffman coding with codes capped at 8 bits, and groups rare exponents under one escape prefix followed by an 8-bit literal. The escape matters: without it, covering all 256 possible exponents under an 8-bit cap would force every code to be 8 bits long. These are estimates for the paper's modelled distributions; the allocated cost it reports lands near 3.0 to 3.2 bits per exponent for matched models in the default CLEAN configuration. The codebook is precomputed from a distribution the user picks, with a small shift sampled from up to 4096 elements to follow the tensor's scale, so no codebook is built during training.

Compressing is easy; compressing in parallel on a GPU is the hard part. In the default CLEAN configuration, exponents are grouped into blocks of 65,536 (the MEDIUM and HIGH settings use 32,768), and each block is cut into 256 independent streams so that 256 pieces of work can run at once. Neighbouring values are dealt to different streams in round-robin order, so one stream does not receive a whole row that may be strongly correlated. Picture 256 baggage belts, each with a fixed length. If a belt's bags carry too many OTHER stamps and the codes do not fit, the leftover destinations spill onto an overflow cart stored uncompressed, and a marker records where the spill starts. A fixed length per belt means every thread knows where to write without waiting for the others.

Training adds a second problem: every tensor is rewritten every step, so the cost is the whole encode-decode round trip through HBM, not a single decode at load time. For a plain SGD update, W_t+1 = W_t − ηM_t, decoding both tensors, doing the arithmetic and re-encoding costs roughly 4N bytes of memory traffic per encode-decode cycle for an N-element tensor, and the paper estimates about 16N bytes for a naive SGD update. LCT's fused kernels read the compressed inputs, compute the result and write the new exponent and sign-fraction bytes in one pass, then compress the exponents, for an estimated 8N bytes. Matrix multiplies are the exception: the paper reconstructs compressed operands as dense inputs because fused encode-decode matmuls did not give a significant speedup.

Here is the idealized arithmetic with numbers held fixed. A BF16 value stored as 8 sign-fraction bits plus about 3 exponent bits takes 11 bits instead of 16, a ratio of 16 / 11 = 1.455×, which is also the paper's best reported ratio. The default CLEAN setting is bounded at 1.4382×, or about 11.1 bits. Now apply it to a model whose parameters take about 8 GiB in BF16: at the best-case ratio (illustrative) that is 8 × 11 / 16 = 5.5 GiB, saving about 2.5 GiB per 8 GiB tensor. The measured result is smaller than a best-case sum because activations, buffers and the shared overflow buffer also count: at sequence length 500 on Qwen3-4B, peak memory fell from 19,439 MiB to 15,142 MiB, a saving of about 4.2 GiB (22.1%).

Setup	Peak memory, baseline → LCT	Step time, baseline → LCT	Source
NanoGPT, 162M parameters (activations compressed)	16,058 MiB → 12,103 MiB (24.6% lower)	8.411 s → 9.013 s (7.2% higher)	arXiv 2610.22179
Qwen3-4B full fine-tune, sequence 500	19,439 MiB → 15,142 MiB (22.1% lower)	3.759 s → 4.058 s (8.0% higher)	arXiv 2610.22179
Qwen3-4B full fine-tune, sequence 1000	21,328 MiB → 16,262 MiB (23.8% lower)	4.006 s → 4.273 s (6.7% higher)	arXiv 2610.22179

Read the "3.5× longer context" claim carefully. The uncompressed baseline ran out of memory at every tested sequence length above 1000 tokens, while LCT completed up to 3500, so 3.5× is the ratio of the longest tested lengths at batch size 1, not a measured ceiling. Exactness also has a cost: step time rises by about 7 to 8%, the codec currently targets BF16 only, the user must supply a distribution that fits the tensor (a mismatch raises storage), and the training code needs explicit integration. On NanoGPT the final validation loss was 3.27978 with LCT and 3.27981 without, which the authors attribute to nondeterministic rounding. The paper names decoding inside matmul kernels and combining lossless with lossy compression as future work.

Goes deeper in: GPU & CUDA → Memory Hierarchy → HBM: Where Your Model Lives

Related explainers

vLLM v0.20 — TurboQuant 2-bit KV cache — the lossy way to shrink the same memory
Nemotron-H FP4 block scaling — lowering bits per value by changing the number format

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based