Four claims behind 138 sparse-autoencoder studies, and the test each one needs
A 28-page critical survey of 138 sparse-autoencoder studies separates recoverability, semantic interpretability, mechanistic faithfulness and behavioral utility, and shows how prompt templates, baselines and layer coverage move the published numbers more than the dictionaries do.
A patch can lift a model's output while the computation behind that output stays where it was.
In a three-coordinate toy model, coordinate 1 tracks the input while the output ignores it, coordinate 2 reaches the output while staying dormant, and coordinate 3 carries the route the model actually takes. Patch coordinate 1 into coordinate 2, and the answer comes out right through a path the network never took. 1
Three coordinates and two output values are enough to see it, and this counterexample sits at the centre of how sparse autoencoders get validated.
That example looks exactly like the evidence SAE papers offer for a mechanism claim. A sparse autoencoder learns a wide dictionary from a layer's activations and reconstructs each activation from a handful of entries. The promise behind the method is that a dictionary entry names something the model computes with. 2
Sparse Autoencoders for Mechanistic Interpretability in NLP reviews 138 studies that make some version of that promise. Salima Lamsiyah, Adelaide Danilov, Oleksandr Marchenko Breneur and Aria Nourbakhsh, at the University of Luxembourg, posted it to arXiv on 14 October 2026; it runs 28 pages with five figures, and it is a preprint that no venue has refereed. 3
The survey separates four claims that SAE papers tend to merge — recoverability, semantic interpretability, mechanistic faithfulness and behavioral utility — and gives each claim its own test. 4 That separation does most of the work, because where the field looks divided the review keeps finding studies that measured different things under settings that changed the answer.
What two SAE papers must share before their numbers can be compared
The review records each study as a triple: dictionary design, feature operation and NLP function. Design covers how the feature object is obtained — sparsity allocation, geometry, training objective and coverage, inference and activation site. Operation covers what is done with it: characterize, predict, trace dependencies, edit or construct. Function names the objective that defines success. 5
Alongside the triple sits a qualifier band the review treats as part of the comparison: model and version, activation site, training data, supervision, feature selection and intervention protocol. Two studies that agree on all three facets can still differ inside that band, and the difference moves the result.
Recoverability belongs to the whole pipeline
The first claim is that the dictionary recovered the structure the study was after. Recoverability has separable bottlenecks, and the review treats them as four: the representation and its geometry, sparse allocation, training coverage and inference error. 7
Splitting and absorption complicate the first step. A concept can be spread across several latents, and a specialised latent can absorb a broader property, so an inactive coordinate carries no information about whether the underlying information is present. 8
Allocation sets the second. In a synthetic dictionary with 128 generating features in 42 dimensions and an 8,192-unit SAE, TopK recovery at an alignment threshold of 0.9 runs 98.4% at K=20, 24.2% at K=30 and 23.4% at K=50. Group Bias Adaptation reaches 100% when its target-frequency groups cover the generating frequency, and 3.9% when the groups span 0.001 to 0.01 and exclude it. 9
Those figures come from a model with a known ground truth, and the review keeps them there: they show sensitivity to allocation and coverage, and they say nothing yet about long-tail linguistic recovery. 10
The review's matching rule for dictionary comparisons follows from the same numbers: hold average sparsity, data and insertion protocol fixed, and record decoder normalisation, because it changes coefficient scales.
The allocation rule a study chose also resists ranking by reconstruction alone. BatchTopK improves reconstruction over TopK, while its Gemma-2-2B comparison with JumpReLU comes out mixed on downstream cross-entropy degradation. 10
Four claims, and the test each one needs
The review's second contribution is an evidence ladder that grades claims, with no score attached to a paper. A feature description supported by selected activating examples sits at the weak end; one that predicts activation on held-out positives and negative examples sits in the middle; one tested against hard negatives and neighbouring features at the strong end. 11
The ladder matters most for the third claim, mechanistic faithfulness — the claim that an explanation identifies computation the original model uses. Here the review assembles the same counterexample and asks what a successful intervention can carry. 12
The paper that supplies the decomposition is Makelov, Lange and Nanda's study of interpretability illusions in subspace activation patching, which reported the failure directly. 14 The survey's addition is to make it a general requirement: a mechanism claim needs a natural-pathway test on top of an intervention.
Control design decides whether an edit measures the feature at all. Replacing an activation with its decoded reconstruction also removes the reconstruction error, so the measured effect mixes the feature edit with reconstruction damage. The review's error-preserving version keeps that residual, which makes a null edit return the original activation and leaves the feature delta as the only change. 15
Two controls it asks for beyond that are cheap and rarely reported. A randomised host model tests whether an interpretability metric distinguishes learned computation from structure that arrives without training; automated interpretability metrics scored similarly on trained and random transformers in the study that ran it. 16 A descriptive-collision test asks whether one feature description also fits unrelated features. 17
The ladder turns into a table anyone can apply to a paper they are reading.
| Claim | Evidence that supports it | What the review counts as missing |
|---|---|---|
| A feature description identifies a distinct feature | Predictions confirmed on held-out positives and negatives, with hard negatives | Discrimination from neighbouring features 11 |
| A feature predicts or diagnoses an outcome | Held-out prediction against a relevant baseline | Matched supervision, selection and label budget 15 |
| A feature plays a mechanistic role | Targeted intervention producing the predicted effect | Semantic counterfactuals, reconstruction controls, natural-pathway evidence 12 |
| SAE features control a task outcome | Improvement over a baseline with effects on other capabilities checked | Matched feature selection and control baselines, robustness and persistence 11 |
Why the studies disagree
Take the fourth claim, behavioral utility, and ask whether SAE features help on a task. The review's comparisons make the protocol effect visible, and two of them are worth carrying around.
Prompt template first. In a Gemma-2-2B sentiment circuit, the unmodified model scores 45.29, 68.53, 80.33 and 86.43 F1 across four SST-2 templates, and the same circuit intervention scores 82.48, 83.52, 86.61 and 89.39. The gains run 37.19, 14.99, 6.28 and 2.96 percentage points. 18
A 34-point spread from the prompt template alone sets the scale for any circuit gain measured this way. The review reports the three-run means without dispersion, and it declines to attribute the spread to baseline headroom, since that would need its own test. 19
The second comparison puts an SAE steering direction against simpler alternatives in one benchmark. On CLaS-Bench over 32 languages with Llama-3.1-8B-Instruct, the mean language-steering score is 84.5 for residual difference-in-means, 80.1 for LAPE, 67.7 for an English prompt, 48.6 for a linear probe, 42.3 for SAE difference-in-means, 23.6 for LDA and 15.1 for PCA. 20
The settings decide how much of the gap belongs to the dictionary. The residual method uses layer 20 and the SAE method layer 25, both at strength 5; SAE checkpoints exist only at layers 4, 12, 18, 20 and 25; the SAE intervention adds the reconstruction residual back after decoding. 21 The 42.2-point gap therefore describes the two selected pipelines, with dictionary quality, layer coverage and selection procedure still entangled.
Baseline construction does the same work in classification. Sparse probes lose much of their apparent advantage once attention-pooled dense probes enter the comparison, in the study that set out to test whether SAEs are useful for probing at all. 23 Both classification studies the review compares use pooling, which leaves the rest of their difference unaccounted for.
Information access splits the retrieval results the same way. A detector can flag an unsupported answer without explaining it, a source-arbitration method changes which evidence the model relies on, and a repair method can restore an answer when a known correct token is available.
Optimising latents toward that known token on roughly 255 distractor-vulnerable examples per model establishes conditional repair capacity; autonomous detection and correction is a different measurement. 24
Unlearning brings its own denominators. The WMDP-Bio results the review inspects count questions answered correctly under all 24 answer-option permutations — 172 of 1,273 for Gemma-2B-it — which measures reliably available knowledge on a denominator narrower than the full benchmark population. 25
What the 138 studies are made of
The review's corpus runs from 199 records to 138 included studies: five duplicate or overlapping version pairs merged into 194 unique candidates, then scope and source verification. Searches covered sparse autoencoders, dictionary learning, mechanistic interpretability, language models, feature circuits and activation steering, with a literature cutoff of 12 October 2026. 26
Its multi-label counts show where the studies it includes spend their effort. Across all 138 studies, contributions that trace dependencies never exceed 8 in any function row, while characterising an adopted dictionary and editing or constructing with one dominate: the adopted/comparative design row alone holds 64 characterising and 55 edit-or-construct contributions. 27
That distribution accounts for the thin evidence behind the comparisons above. Most published work adopts an existing dictionary and then applies it, so the audits that would settle recoverability and pathway fidelity are the rarest contributions in the corpus.
The cohort years trace how the surveyed literature moved. Counting studies by reference year, dictionary design runs 1 in 2023, 5 in 2024 and 12 in 2025; feature methodology runs 0, 4, 13 and 16 across the same years; circuits and reasoning run 0, 5, 6 and 9.
Speech and multimodal analysis goes from 1 in 2025 to 11 in the partial 2026 window, and dictionary design drops back to 10 there. 28 The review labels the count a cohort profile of the studies it included, with the 2026 window still partial.
Choosing the comparator before the dictionary
The review's practical section answers the question its comparisons keep raising: when does an SAE earn its cost? 29
The ordering it recommends runs from cheapest to most expensive. For a known predictive target, start with a well-tuned supervised probe and a matched label budget.
For output control, try direct activation directions before paying for dictionary training and feature selection. For a localised computational hypothesis, neurons, heads or a functional surrogate may supply the unit you need. An SAE becomes the right instrument when a reusable, fine-grained vocabulary is what the work requires.
Five questions turn that into a test of any SAE paper you meet.
- Was the recovery target stated, and were allocation, coverage and sparsity matched? Recovery moves from 98.4% to 23.4% across K=20 to K=50 in a known synthetic model. 10
- Does the feature description survive hard negatives? A description that fits selected examples has cleared the weakest rung. 11
- Does a mechanism claim come with a natural-pathway test? A successful patch satisfies the manipulation check and leaves the pathway open. 1
- What baseline did the utility claim beat? A dense probe, a linear probe and a direct activation direction are the comparators that change published conclusions. 23
- Which denominator does the gain use? Selected populations and known-answer conditions measure a narrower capacity than full-benchmark accuracy. 25
Where the evidence stops
The survey is a preprint, and its comparisons re-plot results from other papers; the authors report no new model experiments of their own. 3 Its authors say plainly that architecture, data, feature representation and evaluation protocol differences limit cross-study comparison, so the analyses report the conditions under which a finding holds. 30
The 138-study corpus yields no ranking of dictionaries, and the review explains the omission: the comparisons it surveyed have not been matched. A reader hunting for "the best SAE" will close the paper still hunting.
A headline number on how much SAE features improve downstream systems is what I went looking for, and what the paper offers instead is a set of comparisons that each hold their own setting. The corpus-wide counts are the review's own extraction, checkable against the register it publishes alongside the paper. 26
What the survey settles
Two separations survive the detail. The first is between recovering a structure and explaining a computation: a dictionary can reproduce activations well and still leave the model's actual route unvisited.
The second is between a method's effect and the protocol around it: a prompt template alone moves a circuit gain by 34 points, and layer availability accompanies a 42.2-point gap between two steering directions.
For a reader deciding whether the 28 pages are worth an evening, the return is a rubric and a set of comparison habits; the verdict on any one method stays with the reader.
Sections 3 to 6 carry the rubric, Table 1 grades claims, and the appendices hold the case-by-case arithmetic. Someone building with SAEs will get more from section 6's ordering and the five questions than from any single number in the paper.
The full text, with the taxonomy codebook, the evidence ledger and the secondary analyses of published results, is at Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility.
References
- 1
- 2Bricken et al., Towards Monosemanticity
transformer-circuits.pub
- 3Lamsiyah et al., abstract page
arxiv.org
- 4
- 5
- 6
- 7
- 8Chanin et al., A is for Absorption
proceedings.neurips.cc
- 9Chen et al., Taming Polysemanticity in LLMs
proceedings.iclr.cc
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17Lamsiyah et al., Section 4.1
arxiv.org
- 18
- 19Lamsiyah et al., Appendix F.1
arxiv.org
- 20Gurgurov et al., CLaS-Bench
aclanthology.org
- 21Lamsiyah et al., Appendix F.2
arxiv.org
- 22Lamsiyah et al., Figure 5
arxiv.org
- 23
- 24Giannetti et al., The Mechanics of Interference
aclanthology.org
- 25
- 26
- 27
- 28
- 29
- 30Lamsiyah et al., Limitations
arxiv.org
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.