AI Explained
Plain explanations of trending AI concepts, with live visualizations.
BF16 vs FP16 in RL post-training — Weight sticking and stochastic rounding — What does it mean?
Round-to-nearest hides tiny RL updates from BF16 weights; stochastic rounding lets them through without FP16 or a higher learning rate.
KernelBench study of a local 7B model — Correct first, fast second — What does it mean?
A 7B model's CUDA kernels only tie cuBLAS; real wins are fused epilogues, and best-of-n under a real oracle beat more agent structure.
NVLinkOneSided speeds up MoE all-to-all — One-sided writes vs count exchange — What does it mean?
MoE GPUs push tokens straight into their own slots on peer GPUs, cutting the fixed costs that dominate small-batch decode.
FP4 Pretraining Study — Rare-fact acquisition vs validation loss — What does it mean?
NVFP4 matched BF16 loss within 0.5% but needed 1.32x the exposures to learn rare facts: the loss average hides the long tail.
FOSCAR speeds up 2-bit KV decode — Affine folding of KV dequantization — What does it mean?
Low-bit KV caches saved memory, not time; FOSCAR applies each group's scale and offset once, not per value, so 2-bit decode gets fast.
TriAttention paper — Sparse mixed-precision attention tiles — What does it mean?
One calibrated policy per head decides which attention tiles to skip, run in 4-bit, or run in 8-bit.
KOSMOS — Self-scheduling persistent kernels — What does it mean?
KOSMOS lets GPU workgroups switch between moving data and computing by readiness, replacing a fixed split that leaves idle windows.
ZYFA FP8 attention on AMD MI355X — power-of-two probability scaling and ping-pong softmax — What does it mean?
FP8 doubles matrix speed but not softmax; ZYFA shortens and overlaps softmax work.
Blackwell locality domains — Die-aware memory placement — What does it mean?
Blackwell's two dies make memory non-uniform. Pinning data to one die speeds up some accesses and slows others; the access pattern decides.
Co-design linear attention for TPU systolic arrays — Systolic Linear Attention — What does it mean?
Gated-delta linear attention rebuilt around the TPU's 256×256 matrix unit, so tiles are full and the layer runs as one fused kernel.
LARK formally verifies LLM-written GPU kernels — Segment-wise Lean equivalence proofs — What does it mean?
LARK has LLM agents prove in Lean that a GPU kernel equals its PyTorch reference, one segment at a time.
Sonar-MoE frees MoE expert gradients mid-backward — Optimizer-in-backward for experts — What does it mean?
Sonar-MoE updates each MoE expert tensor during backward and frees its gradient, cutting peak memory 19–22%.