AI Explained
Plain explanations of trending AI concepts, with live visualizations.
AMD RCCL and rocSHMEM — GPU-initiated SDMA copy engines — What does it mean?
A GPU kernel can now hand collective copies to DMA copy engines, so 1 compute unit replaces 64 at matching bandwidth.
NVIDIA open-sources Snapshot — Checkpoint-restore vs cold start for inference replicas — What does it mean?
A new inference replica spends minutes compiling and capturing CUDA graphs; Snapshot saves the ready process and restores it in seconds.
Goal Encoding in LLMs — Non-Recurrent Goal-Seeking — What does it mean?
An LLM keeps a goal by re-reading the task in its prompt each step through a few goal heads; no state survives token to token.
LazyWeights paper — Decoding weights between DRAM and L2 — What does it mean?
LazyWeights unpacks Huffman-coded weights in a unit between DRAM and L2, so only coded bytes cross the bus and kernels stay unchanged.
SignRot for 4-bit LLM quantization — Sign-adjusted Hadamard rotation — What does it mean?
One giant activation splits rotated 4-bit values into two clusters; SignRot flips signs to merge them and a zero-point frames the result.
Cited but Unused — Counterfactual value substitution — What does it mean?
A cited passage that agrees with a claim does not prove the model used it. Edit the passage and re-ask to find out.
Phrase-Level Speculative Decoding — Drafting in phrases, not tokens — What does it mean?
A tiny draft head guesses whole phrases from its own vocabulary, so one accepted guess banks several target tokens.
WebTrap Farm — Paired usability calibration — What does it mean?
A failed attack is only a defense if the agent could do the benign twin of the task. WebTrap Farm scores web agents that way.
BF16 vs FP16 in RL post-training — Weight sticking and stochastic rounding — What does it mean?
Round-to-nearest hides tiny RL updates from BF16 weights; stochastic rounding lets them through without FP16 or a higher learning rate.
KernelBench study of a local 7B model — Correct first, fast second — What does it mean?
A 7B model's CUDA kernels only tie cuBLAS; real wins are fused epilogues, and best-of-n under a real oracle beat more agent structure.
DeepSeek-V4-Flash KV reuse study — Cross-version KV cache reuse — What does it mean?
An updated model can reuse its predecessor's KV cache for 2-5x faster TTFT, but correct answers and faithful behavior need separate tests.
SinkState — On-the-fly low-rank recurrent state — What does it mean?
Hybrid-model recurrent states are nearly low-rank; SinkState keeps them as online SVD factors, cutting state memory up to 2.6×.