AI Explained
Plain explanations of trending AI concepts, with live visualizations.
SignRot for 4-bit LLM quantization — Sign-adjusted Hadamard rotation — What does it mean?
One giant activation splits rotated 4-bit values into two clusters; SignRot flips signs to merge them and a zero-point frames the result.
Phrase-Level Speculative Decoding — Drafting in phrases, not tokens — What does it mean?
A tiny draft head guesses whole phrases from its own vocabulary, so one accepted guess banks several target tokens.
DeepSeek-V4-Flash KV reuse study — Cross-version KV cache reuse — What does it mean?
An updated model can reuse its predecessor's KV cache for 2-5x faster TTFT, but correct answers and faithful behavior need separate tests.
SinkState — On-the-fly low-rank recurrent state — What does it mean?
Hybrid-model recurrent states are nearly low-rank; SinkState keeps them as online SVD factors, cutting state memory up to 2.6×.
Bursts That Outlast Themselves — Prefix-Cache Metastability — What does it mean?
A short burst can trap a prefix-cached agent server at a 10% hit ratio; parking admitted sessions by KV demand gets it out.
Fisher quantization screen paper — Predicted KL vs quantize-and-measure — What does it mean?
One fp32 gradient pass predicts each weight format's damage to a model's outputs, so layer formats are chosen without testing every plan.
AES paper — Entropy-sparing vs uniform length penalty — What does it mean?
AES penalizes only late high-entropy tokens in correct answers, not every token, cutting reasoning about half without losing accuracy.
Lost in Rounding — Stochastic rounding for BF16 recurrent state — What does it mean?
BF16 round-to-nearest freezes small updates to a hybrid model's state and lengthens reasoning; stochastic rounding fixes it at 16 bits.
StrataFlux — Selective precision streaming — What does it mean?
Keep low-bit weights and KV on the GPU; fetch extra precision bits from RAM or SSD only for the channels and tokens that matter now.
LowRAM KV cache compression — Predicting the value cache from keys — What does it mean?
Inside one attention head, values are mostly predictable from keys, so LowRAM stores a small predictor plus fixes for the worst tokens.
The Gating Mechanism of KV Cache Eviction — The constant-gate no-op — What does it mean?
Softmax cancels a gate that is equal for every key, so a learned KV eviction gate trained on LM loss learns nothing.
Non-prefix KV reuse study — Targeted KV repair — What does it mean?
Where to spend a small KV-repair budget when stitching cached chunks: on the chunk that holds the answer.