AI Explained
Plain explanations of trending AI concepts, with live visualizations.
A Circuit for In-Context Learning — gated associative lookup — What does it mean?
Few-shot learning in 3–4B transformers traced to key-value label slots, reader heads, a repeat gate and brake heads.
DepthGraft rebuilds deep-layer KV from shallow and middle layers — Cross-layer KV reconstruction — What does it mean?
DepthGraft rebuilds the last third of layers' KV from earlier layers, cutting the cache by about a third.
Library of Layers — Dynamic layer ordering from a shared block library — What does it mean?
A router picks Transformer blocks from a shared library per input; library size slides between looped and MoE.
Who Should Score the Cache paper — Mid-depth layer scoring — What does it mean?
One mid-depth layer scoring the whole KV cache keeps answer tokens that shallow layers wrongly evict.
UpLoop trains looped Transformers at no extra pretraining compute — Upcycling a shallow Transformer into a looped model — What does it mean?
Train shallow, copy layers to grow, then loop a shared middle block, all at the dense model's FLOPs.
Latent Sparse Attention at 10M tokens — Hierarchical key clusters — What does it mean?
LSA clusters keys into a learned tree so queries score O(log T) nodes, not every key: 22.7x faster prefill layer at 10M tokens.
Flide answers decision queries 9.5× faster than vLLM — ForkAttention exact state forking — What does it mean?
Flide runs a shared state once and forks each question from it, so decision models answer faster with outputs identical to asking alone.
MERLIN — Second-moment routing for multi-state linear attention — What does it mean?
MERLIN weights linear-attention memory segments by both the mean and the variance of the key-query match.
Multi-View MoE — Multi-router expert selection with latent input views — What does it mean?
Several routers pick MoE experts, then their score ratios blend learned views into each expert's input, with no extra expert FLOPs.
Every Token Needs Its Own Teacher (MTA-OPSD) — Per-token teacher selection — What does it mean?
MTA-OPSD lets every token pick its own teacher during on-policy self-distillation, not one fixed teacher per response.
Decode-length prediction hits a noise floor — Irreducible sampling error — What does it mean?
No prefill-side predictor can beat a noise floor set by sampling randomness; benchmarks must fix model, data and sampling.
TextSeal2 watermarks LLM text through speculative decoding — Exponential-race sampling — What does it mean?
A keyed exponential race samples each token and marks it; TextSeal2 merges keys to keep detection strong.