GateWatch — Boolean circuit probes over SAE features — What does it mean?
The news. On November 18, 2026, Firas Spanioli and Yaniv David of the Technion posted GateWatch (arXiv 2611.24619). Across 24 classification tasks on Gemma-2-9B with Google's pretrained Gemma Scope SAEs, probes limited to five gates reached 80.49% mean test accuracy with 3.8 features on average, against 81.66% for a probe over all 16,384 SAE features, while using 99.98% less probe memory. A vLLM plugin, GateWatchServe, ran 1,000 concurrent probes at 97–98% of baseline throughput. Read the paper →
Picture the alarm panel in an office building. Nobody connects the alarm to a camera feed and asks a computer to score every pixel; the panel reads a handful of sensors, each of which is simply tripped or not, and a wiring rule decides. GateWatch builds a model monitor the same way: a few on/off features from a sparse autoencoder, combined by a rule small enough to read aloud. One complete rule from the paper, for telling 5-star Amazon reviews from the rest, reads: 5 stars unless the review shows repeated frustration AND (accountability OR (negative claims AND service problems)). It uses four features and reached 95.49% accuracy on that task, against 89.44% for the full 16K-feature SAE probe.
The usual SAE probe works like the camera wall. It is a logistic regression: each feature gets a weight, and the weighted values are added into one score. With hundreds or thousands of nonzero weights there is no short reason for a verdict, and a sum cannot express some rules at all, such as XOR ("exactly one of these two concepts is present"), which a Boolean circuit expresses with one gate. GateWatch offers three ways to find a circuit under a fixed gate budget. Compile trains a logistic regression over the firing bits, converts its rounded rule into logic gates with chip-design synthesis tools (Yosys and ABC), then removes gates while validation accuracy holds. Search uses symbolic regression to look for a short formula directly. Train learns the gates by gradient descent, using a differentiable logic-gate network whose soft gates are rounded to real ones at the end. Across the 24 tasks, Search, Compile and Train gave the most accurate compact probe on 12, 8 and 4 tasks respectively.
The panel never re-traces its wiring when a sensor trips; it reads the answer off the printed chart. At serving time GateWatch turns each circuit into a truth table, so a probe's verdict is one table read instead of a pass through gates. A circuit with at most five two-input gates depends on at most six input bits, so its table has 26 = 64 entries and fits in one 64-bit integer. GateWatchServe runs in three stages inside vLLM, alongside the model's own forward pass. First, at each layer a probe reads, it multiplies the residual stream by only the encoder rows that some probe needs, applies thresholds, and packs the bits; each feature is computed once and shared by every probe that uses it, which is the shared set of sensors. Second, it updates small per-request token rules: LAST (fired on the latest token), ANY (fired at any token so far) and COUNT2 (fired at least twice). Third, after the last layer, one batched step looks up every probe's table. Only bits and small counters persist between tokens, so no full hidden states are buffered.
Where the memory goes (worked example). Hold the model fixed at Gemma-2-9B, whose residual width is d = 3,584, and the probe at the paper's five-gate average of 3.8 features. Each feature needs one encoder row stored in BF16, which is 2 bytes per number: 3,584 × 2 = 7,168 bytes, plus 8 bytes for its bias and threshold, so 7,176 bytes per feature. Multiply by 3.8 features and you get about 27,300 bytes, or about 26 KiB per probe (the paper's measured mean is 26.3 KiB). The rule itself is the 8-byte truth table, under 0.1% of the total. Compute has the same shape: 3.8 × 3,584 ≈ 14,000 multiply-adds per token, against 16,384 × 3,584 ≈ 59 million for one layer of the full 16K-feature encoder. Set 26.3 KiB beside the full SAE probe's 115,261 KiB and you get the paper's headline: about 4,380× less memory per probe.
| Probe type | Mean accuracy, 24 tasks | Memory per probe | Readable rule | Source |
|---|---|---|---|---|
| Best dense activation probe | 83.84% | 987.4 KiB | No | Table 2 |
| Full 16K-feature SAE probe | 81.66% | 115,261 KiB (about 113 MiB) | No | Table 2 |
| GateWatch, Search, at most 5 gates | 80.49% | 26.3 KiB | Yes | Table 2 |
A defense-in-depth stack wants many narrow detectors, but every byte a monitor keeps on the GPU is a byte the KV cache cannot use, the same squeeze that adapter rank puts on multi-LoRA serving. The paper stress-tested 1,000 concurrent probes reading all 42 layers and all tokens. Pooled MLP probes (small neural networks over the hidden state) and SAE probes fell to 66–81% of baseline throughput and 68–72% of KV-cache capacity, while GateWatch probes stayed at 97–98% throughput and at least 99.8% KV capacity, because all 1,000 together hold only 26–30 MiB of encoder rows and 0.98 KiB of state per request. Under live traffic at 8 requests per second, p99 decode latency rose 5.5% with GateWatch probes, against 23.7% for MLP probes and 95.5% for SAE probes.
The readable rule is not free. The best dense activation probe still averages 83.84%, about 3.4 points above GateWatch's 80.49%. Against the full SAE probe, the 1.17-point mean gap is not statistically significant (95% confidence interval −3.6 to +1.3 points), and the spread is wide: GateWatch matched or beat it on 10 of 24 tasks and lost on 14. Part of GateWatch's accuracy also comes from reading more of the model. Limited to the baselines' own access (layer 20, last token only), five-gate Search probes trail the SAE probe by 7.64 points, and reading all layers and tokens recovers most of that gap. In return, the authors report that the readable rules exposed dataset shortcuts that are hard to see in a dense probe. All results are on one model, Gemma-2-9B with Gemma Scope SAEs; the paper does not show whether the same compactness holds on other model families.
Goes deeper in: Agent Engineering → Layered Guardrails → Defense-in-Depth for Agents