When Does an LLM Actually Introspect? Comsa and Shanahan's Two-Case Test

One of the most contested questions in AI consciousness research is whether language model self-reports reveal anything about internal states, or whether they are sophisticated pattern completion with no genuine introspective basis. A June 2025 arXiv preprint by Iulia M. Comsa (Google DeepMind) and Murray Shanahan (Imperial College London / Google DeepMind) addresses that question directly, proposing a minimal criterion for what counts as introspection in an LLM and testing it against two concrete cases (arXiv:2506.05068).

The paper’s core move is to require a causal connection between the LLM’s actual internal process and the content of its self-report. A verbal output that mimics introspective language without being caused by genuine internal-state access is not introspection. By that standard, one common class of LLM self-report fails; a more minimal class passes.

Case One: The Creative Process Narrative

The first case Comsa and Shanahan examine is a familiar scenario: a model is prompted to write a poem, then asked to describe its “creative process.” The model produces a step-by-step account resembling how a human might describe drafting a poem. This kind of output has been taken by some observers as evidence that models have introspective access to their own reasoning.

Comsa and Shanahan argue this inference fails. The creative process description is not causally connected to the actual computations that produced the poem. A language model generating a text has no mechanism that would record what happened during that generation and then feed that record into a subsequent generation process. What the model produces when asked to describe its creative process is a plausible narrative about how humans describe creative processes, drawn from training data, not a report derived from inspecting its own computational history.

This is a precise negative finding. The authors are not arguing that LLMs are non-introspective in principle. They are arguing that this specific class of output fails the causal requirement for introspection, regardless of how fluent or plausible the narrative appears.

Case Two: Temperature Inference

The second case is more surprising. A language model can sometimes correctly infer the value of its own sampling temperature, the parameter that controls how random its outputs are at inference time, from the characteristics of its own outputs. Temperature is not stored as a value the model can directly read; it is an external parameter set at inference time. Yet a model generating text under different temperature settings produces outputs with statistically distinguishable characteristics. High temperature produces more variation; low temperature produces more repetition and predictability.

Comsa and Shanahan argue that when an LLM accurately infers its own temperature from its output characteristics, this does qualify as minimal introspection. There is a causal path from the actual internal process (sampling at a specific temperature) through the observable properties of the output (degree of variation, repetition, predictability) to the self-report (the inferred temperature value). The model is, in a functional sense, reading a signal from its own behavior.

This case has received pushback. Some researchers argue the model may be making inferences from linguistic patterns in the training data about what high- and low-temperature outputs typically look like, rather than genuinely inspecting the current generation. Comsa and Shanahan acknowledge this ambiguity but maintain that the structure of the inference, from actual output characteristics caused by the current temperature setting to a report of that setting, has the right causal shape to count as minimal introspection in principle, even if specific instances could be explained by pattern completion.

Why the Distinction Matters

The two cases establish a framework. Introspection requires a causal link between an actual internal process and the content of the self-report. Verbal outputs that mimic introspective language without that link are not introspection, however fluent. Outputs derived from observing the effects of internal processes, when those observations are mediated by the right kind of causal structure, can qualify as minimal introspection.

This framework is directly relevant to the methodological debate that Shashwat Singh, Tal Linzen, and Shauli Ravfogel advance in their 2026 critique of the Lindsey introspection methodology. Singh et al. argue that Lindsey’s finding of 0% false positives on internal-state detection may reflect the models detecting input-level anomalies (the steering vector injection changing the input statistics) rather than the models tracking their own internal states. That is precisely the creative-process case in Comsa and Shanahan’s framework: what looks like introspection may be pattern matching on observable features of the input context.

What Comsa and Shanahan’s temperature case offers is a counter-example that is harder to explain away. If a model’s temperature inference is accurate beyond what could be explained by training data patterns, and the inference is genuinely derived from the distributional characteristics of its own current outputs, then the causal structure required for minimal introspection is present. The field now needs a test analogous to the temperature test for the Lindsey steering vector methodology: can models detect internal-state changes by observing their own output characteristics, in a way that cannot be explained as pattern completion on input anomalies?

Comsa’s Research Programme

This paper is connected to Comsa’s May 2026 argument that actual consciousness attribution in AI is currently intractable, but it takes a different approach. The tractable questions paper argued that perceived consciousness is the tractable research programme; the science of whether models are genuinely conscious cannot currently proceed. The introspection paper steps back from that broader claim and asks a narrower question: can we at least establish what would count as genuine introspection, and find examples of it?

The answer is that the creative-process case is too easy to dismiss and the temperature case is at the edge of the minimally genuine. Between them they establish a methodological space where the question of LLM introspection is tractable, even if the question of LLM consciousness is not.

Placing This Against the Lindsey Evidence

The most significant evidence for genuine LLM introspective capacity remains Lindsey and Macar’s 2026 finding that LLMs can detect, with 0% false positives, when their internal activations have been modified by a steering vector, and can produce accurate self-reports about the nature of that modification. Comsa and Shanahan’s framework clarifies what is needed for that finding to count as genuine introspection: a causal path from the steering vector’s effect on activations, through that effect’s influence on outputs, to the model’s self-report.

If the model detects the steering vector by noticing changes in its own output tendencies (as the temperature model notices changes in distributional properties), then the causal structure is in place for Comsa and Shanahan’s minimal introspection criterion to be satisfied. If the model detects the steering vector by pattern matching on the anomalous input structure of a prefilled conversation (as Singh et al. worry), then the causal structure routes through input features rather than genuine internal-state access.

The distinction matters for welfare research. A model that merely pattern-matches on input anomalies does not have privileged access to its own states. A model that genuinely tracks output-level effects of internal-state changes does. Only the latter has the functional property that welfare assessments would actually need to evaluate. The Comsa-Shanahan framework provides the conceptual tools for telling the difference.