Objective-Level Recovery Gaps in Mechanistic Interpretability and SAEs

Every empirical proposal to test artificial intelligence for internal world models, self-monitoring, or indicator properties of consciousness rests on a shared methodological assumption that our mechanistic interpretability tools actually recover the computations the intact network uses. When an automated circuit-discovery algorithm extracts a sparse subgraph of attention heads and MLP layers, or when a Sparse Autoencoder (SAE) isolates a “deception”, “uncertainty”, or “self-reference” feature direction, researchers routinely treat a high intervention faithfulness score as proof that the true internal mechanism has been found.

Two major preprints posted in October 2026 subject that assumption to its most rigorous audit yet. In “Objective-Level Recovery Gaps in Mechanistic Interpretability: Context Distortion Can Misrank Circuits” (arXiv:2610.02098, posted October 2, 2026), Chuqin Geng and Xujie Si at McGill University, the University of Toronto, and Mila prove that standard intervention-defined faithfulness systematically prefers equally sized circuits that reproduce the intact model’s behavior worse on held-out prompts, misranking 9.4% to 41.2% of candidate pairs across four established discovery algorithms (EAP, EAP-IG, ACDC, and Edge-SP). In “Sparse Autoencoders for Mechanistic Interpretability in Natural Language Processing: A Critical Survey” (arXiv:2610.20139, posted October 16, 2026), Salima Lamsiyah and colleagues at the University of Luxembourg audit 138 published SAE studies and demonstrate that feature recoverability, semantic interpretability, behavioral steering utility, and mechanistic faithfulness are four distinct properties that routinely diverge.

Together, these two papers extend the measurement-audit cluster we track in van Rooyen’s Cross-Substrate Access Assay, the Tsinghua AwarenessBench study, and Comșa and Shanahan’s critique of introspection and self-report in language models. Before any internal circuit or SAE latent can serve as evidence in the scientific race to define AI consciousness, the evaluation objective itself must be audited for context distortion.

Part 1 Context Distortion and the Objective-Level Recovery Gap (arXiv:2610.02098)

Automated circuit discovery treats mechanism recovery as a search problem over sparse subgraphs $S$ of an intact model $M$. To evaluate whether a candidate subgraph $S$ is “faithful”, the standard protocol executes $M_{S,a}$ by retaining the edges inside $S$ and replacing all excluded signals outside $S$ via an ablation intervention $a$, such as ordinary resampling (patching in activations from a corrupted donor prompt), mean ablation, or zero ablation. Faithfulness is then scored on validation prompts $V$ by how closely the ablated execution matches the intact model’s output distribution, typically measured by negative Kullback-Leibler divergence ($-\mathrm{KL}$), logit difference (LD), or probability difference (PD).

The answer is no, and the mechanism responsible is context distortion. When an intervention replaces signals outside $S$ with donor activations, means, or zeros, the retained components inside $S$ receive combinations of inputs that never occur in the intact model $M$. Retained attention heads and MLPs then recompute their activations on top of those distorted inputs. A candidate circuit $S_2$ that happens to compensate for or cancel out the artifactual ablation noise on validation prompts will receive a higher faithfulness score than a circuit $S_1$ that faithfully implements the intact computation. Because this failure belongs to the scoring function rather than the search algorithm, Geng and Si name it an objective-level recovery gap.

Controlled Edits and Automated Discovery Outputs (EAP, EAP-IG, ACDC, Edge-SP)

The 96 of 100 repair result in Section 5 is the smoking gun of the paper. For 100 persistent cases where validation KL favored the behaviorally worse circuit, Geng and Si left both candidate circuits and their original behavioral scores $Q$ untouched, and selectively restored a small subset of excluded signals from the recipient prompt’s intact execution during the faithfulness evaluation. In 96 of the 100 cases, simply reducing context distortion outside the circuit flipped the validation KL preference back to agree with held-out behavioral truth.

Part 2 Lamsiyah’s Critical Survey of 138 Sparse Autoencoder Studies (arXiv:2610.20139)

While Geng and Si audit circuit subgraphs, Salima Lamsiyah and colleagues audit the second pillar of modern interpretability, Sparse Autoencoders (SAEs), which decompose dense residual-stream or MLP activations into sparse linear combinations over a learned dictionary of feature directions.

Synthesizing 138 NLP studies (accompanied by a reproducible multi-label research register), Lamsiyah et al. formalize every study $s$ through a four-tuple taxonomy.

Here $D_s$ is Dictionary Design (sparsity allocation such as TopK, JumpReLU, or Matryoshka; geometry; training data coverage; activation site), $O_s$ is Feature Operation (representation characterization, probing/diagnosis, dependency tracing, or behavioral steering/editing), $F_s$ is the target NLP Function (multilingual identity, RAG source arbitration, reasoning, safety/unlearning), and $Z_s$ captures the experimental qualifiers (model scale, prompt format, supervision, and baseline controls).

The survey’s central contribution is an evidence rubric that separates four outcomes routinely conflated in AI safety and consciousness-indicator papers.

1. Recoverability. Whether the SAE dictionary actually recovers a known latent variable of the data-generating process. The survey shows (Evidence Case E02) that recoverability depends heavily on feature frequency coverage and capacity allocation; rare or compositional concepts split or merge silently when sparsity penalties shift.
2. Semantic Interpretability. Whether a human-readable or LLM-generated label for a feature makes discriminating predictions on unseen counterexamples (E20), rather than merely summarizing the top-10 activating tokens.
3. Mechanistic Faithfulness. Whether the SAE feature direction identifies a causal variable that the intact model actually uses during normal forward computation.
4. Behavioral Utility. Whether clamping or steering that feature achieves a desired downstream effect in classification, retrieval, or generation.

Across their secondary re-analyses of published results (Appendix D and Appendix F), Lamsiyah and colleagues prove that Behavioral Utility does not imply Mechanistic Faithfulness. An SAE feature can successfully steer a model’s output language or sentiment by pushing activations along an off-manifold or dormant pathway, and direct-space activation steering (E10) or supervised linear probes (E11) frequently match or outperform SAE steering when evaluated against equal-capacity baselines. Conversely, a feature that is genuinely used by the model for next-token generation (such as language identity) can act as a harmful nuisance variable when the objective shifts to cross-lingual semantic retrieval (LangSAE).

What Both Audits Mean for AI Consciousness Indicators

In our review of Evan Ye’s agency gain and encoding gap and our analysis of the Tsinghua AwarenessBench study, we noted that researchers increasingly look inside transformer weights to decide whether a model possesses genuine self-monitoring or merely imitates self-referential text.

Geng & Si (arXiv:2610.02098) and Lamsiyah et al. (arXiv:2610.20139) establish two mandatory methodological guardrails for that entire research program.

• Never accept an ablated circuit’s KL faithfulness score without auditing context distortion. If EAP, EAP-IG, ACDC, or Edge-SP reports that a tiny subgraph of attention heads is “the self-monitoring circuit”, a 9.4% to 41.2% pairwise misranking rate means the ablation protocol may simply have selected the subgraph least disrupted by donor-activation noise.
• Never equate successful SAE feature steering with discovering a native psychological state. Showing that clamping an SAE latent makes a language model claim or deny consciousness establishes behavioral utility (an external control lever), rather than mechanistic faithfulness (that the unperturbed model computes over a unified internal self-representation).

Comparison to The Consciousness AI

This pair of interpretability audits speaks directly to the verification philosophy we enforce in The Consciousness AI repository, documented on our Core Architecture page and Substrate Console.

In our own work, we learned the hard way that structural diagrams and uncalibrated internal metrics can deceive the builder. During our October 15 architectural audit (tcai-research #57 and #63), we discovered that the final thalamic_relay layer of our baseline policy network was included in no active loss term; across 172 of 172 runs its weights changed by 0.0, meaning its output was fixed drift plus noise even though the module sat in the middle of the forward pass. Had we evaluated that module only by whether downstream behavior looked plausible or by an uncalibrated internal correlation, the silent bypass would have remained hidden.

Geng and Si’s distinction between validation faithfulness under ablation distortion and independent behavioral recovery $Q$, alongside Lamsiyah’s separation of representation probing from causal mechanism, matches the multi-gate verification standard we subsequently instituted across our stack.

1. Tectal motor map gate (tcai-research #58). Proving that spatial target selection in the tectal map causally drives the embodied agent’s physical trajectory (95 and 102 of 120 trials).
2. Oscillator rhythm binding gate (tcai-research #59 and #61). Proving via targeted rhythm disruption (preserving network weights while destroying phase regularity) that oscillatory timing is causally required for multi-feature binding (115/120 intact vs 67/120 disrupted across 3/3 seeds).
3. Homeostatic value gate (tcai-research #62). Proving against a frozen-weight control that online reward learning drives nutritive object selection (56/60 vs 28/60).

When we apply Geng and Si’s context-distortion lesson to our own ablations, the next methodological step for our scalar $\Phi$ and workspace gates is clear. Whenever we ablate or clamp a module in The Consciousness AI to measure its causal contribution, we must test recipient-intact context restoration alongside raw zero or noise ablation so that out-of-distribution input shift does not misrank our own internal circuits.

Sources

• Chuqin Geng and Xujie Si. Objective-Level Recovery Gaps in Mechanistic Interpretability: Context Distortion Can Misrank Circuits. arXiv, posted October 2, 2026 (arXiv:2610.02098v1).
• Salima Lamsiyah and colleagues. Sparse Autoencoders for Mechanistic Interpretability in Natural Language Processing: A Critical Survey. arXiv, posted October 16, 2026 (arXiv:2610.20139v1).
• Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. Towards Automated Circuit Discovery for Mechanistic Interpretability. NeurIPS, 2023.
• Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms. COLM, 2024.
• The Consciousness AI Project Repository. Architectural verification gates #57, #58, #59, #61, #62, and #63.

Researchers covered here

• XS Xujie SiDepartment of Computer Science, University of Toronto. Mila - Quebec AI InstituteObjective-level recovery gaps and context distortion in mechanistic interpretability circuit discovery, program synthesis and neuro-symbolic verification 1 article
• CG Chuqin GengSchool of Computer Science, McGill University. Mila - Quebec AI InstituteObjective-level recovery gaps and context distortion in mechanistic interpretability circuit evaluation, neural network verification 1 article
• SL Salima LamsiyahDepartment of Computer Science, University of LuxembourgCritical survey of 138 Sparse Autoencoder (SAE) studies for mechanistic interpretability in NLP, four-outcome separation of recoverability, semantic interpretability, mechanistic faithfulness, and behavioral utility 1 article
• PB Patrick ButlinEleos AI Research. Previously the Global Priorities Institute, OxfordLead author of the indicator properties framework for assessing consciousness in AI systems 24 articles