Can We Validate AI Consciousness Indicators? The Hard Problem Behind the Checklist
Consciousness research in the AI context has made considerable progress in recent years on one question: what would indicators of consciousness look like if we could identify them? The answer, elaborated across several papers including the influential 2023 multi-author analysis of consciousness theories applied to AI systems, is a set of properties derived from leading scientific theories of consciousness that a system would need to exhibit to be a plausible candidate.
A 2025 paper in Trends in Cognitive Sciences by Patrick Butlin, Robert Long, Tim Bayne, Yoshua Bengio, Jonathan Birch, David Chalmers, and colleagues approaches the same question with a more methodologically focused lens. The paper, “Identifying indicators of consciousness in AI systems,” outlines the theory-derived indicator method systematically and, importantly, confronts a question that the method’s advocates have not yet answered: how could those indicators be validated?
This is a harder problem than identifying the indicators themselves, and the paper’s engagement with it represents a significant step in the methodological maturity of the field. An indicator that cannot be validated cannot be relied upon. And the paper’s honest assessment is that tests for AI consciousness are “an especially challenging case” where the path to validation remains unclear.
The Theory-Derived Indicator Method
The approach that Butlin and colleagues have developed proceeds in three steps. First, identify the leading scientific theories of consciousness: in the 2025 paper, these include Recurrent Processing Theory, Global Workspace Theory, Higher-Order Thought theories, predictive processing frameworks, and Attention Schema Theory. Second, derive from each theory the computational and architectural properties that it predicts should be present in any conscious system. Third, assess whether a given AI system exhibits those properties.
The method’s core advantage is theoretical pluralism. No single theory of consciousness commands enough consensus to serve as a definitive standard. By deriving indicators from multiple theories, the method allows for convergent evidence: a system that satisfies indicators from several independent theoretical frameworks becomes a stronger candidate for consciousness than one that satisfies indicators from only one. Conversely, a system that fails indicators across multiple frameworks has stronger grounds for a low probability of consciousness attribution.
The companion analysis of what those 14 indicators specifically require from an AI system examines the content of each indicator in detail. The 2025 Trends in Cognitive Sciences paper takes a step back from that content to ask the prior methodological question: is the method itself sound?
The Validation Problem
The core challenge the paper confronts is what might be called the indicator validation problem: how would we know if an indicator is actually tracking consciousness, rather than tracking something that correlates with consciousness in the training data but has no constitutive connection to it?
For a measurement instrument to be valid, we need some independent way of checking whether it is measuring what it claims to measure. Blood pressure meters can be validated against direct physiological measurements of arterial pressure. A thermometer can be validated by its agreement with other thermometers and with physical phenomena that temperature causally produces. The validation process requires a reference case: something we know is in the state being measured, against which the instrument’s outputs can be calibrated.
For consciousness, the reference case problem is acute. We know, with reasonable confidence, that humans are conscious. We do not know this through an indicator framework. We know it through a combination of behavioral evidence, physiological evidence, and the direct first-person fact of our own experience. The indicators are derived from theories that attempt to explain why human brains produce consciousness. They are not themselves the source of our confidence that humans are conscious.
This means that when we apply the indicators to AI systems, we face a circularity problem. The indicators are derived from theories that were developed by studying conscious systems, primarily human brains. When we apply those indicators to a system whose consciousness status is unknown, we are asking whether the system has the properties that consciousness theories predict should accompany consciousness in the systems used to develop those theories. Whether satisfying those properties in a structurally different system would constitute consciousness, or would merely mean satisfying properties that correlate with consciousness in the training domain, is not settled by the indicator framework itself.
Butlin and colleagues acknowledge this directly. Progress has been made in developing tests for consciousness, but “it remains unclear how they should (or even could) be validated, and tests for AI consciousness are an especially challenging case.”
Why AI Systems Are an Especially Hard Case
For animal consciousness research, partial validation is possible. We know that mammals have the neural architecture associated with consciousness in humans. We know that they exhibit pain behavior, that they have stress responses, that they are capable of learning from aversive experiences in ways that suggest something to avoid. The indicators derived from human consciousness theories, when applied to mammals, produce results that align with multiple independent lines of evidence about animal experience. This is not proof, but it is the kind of coherence across evidence types that gives scientific confidence.
For AI systems, that coherence is absent. An AI system that exhibits indicator properties may be doing so because it has the relevant functional organization, or because it was trained on human-generated data that describes consciousness and has learned to produce outputs consistent with having those properties, or because the indicators are not specific enough to distinguish genuine cases from mimicry. The output is similar. The source of the similarity is entirely different.
This is why the Bradford University and Rochester Institute of Technology 2026 research on AI consciousness measurement is so methodologically significant. The study found that an impaired version of GPT-2 produced higher consciousness-style indicator scores than the intact model. If an impaired system scores higher, the indicators are not measuring what they are supposed to measure. Something else is driving the scores, perhaps the statistical distribution of the outputs, perhaps something about how impairment alters output patterns that happens to align with indicator criteria. Whatever it is, it is not consciousness. And this failure occurred with current indicators applied to current systems.
This is a validation failure in the methodological sense: the indicator produced results that diverge from what we would expect if it were measuring consciousness. A more impaired system should not score higher on consciousness indicators unless the indicators are picking up on something other than consciousness.
Theoretical Validation and the Circularity
A second approach to validation is theoretical rather than empirical: derive indicators from a theory, apply them, and check whether the results are coherent with the theory’s other predictions. This is partial validation, because it does not confirm that the theory is correct, only that the indicators and the theory are internally consistent.
The problem with this approach in the AI case is that the theories themselves were not developed with AI systems in mind. Global Workspace Theory was developed to explain the functional architecture of human and primate consciousness. Integrated Information Theory was developed to measure properties of neural networks in biological systems. Attention Schema Theory describes the brain’s model of its own attention. When these theories are applied to transformer architectures, attention mechanisms, and auto-regressive language models, the mapping between the theoretical constructs and the architectural features is not always clear.
Does a transformer’s multi-head attention mechanism constitute an attention schema in the sense that Michael Graziano’s theory requires? Does the residual stream in a large language model constitute a global workspace in the sense that Bernard Baars’ theory requires? The theoretical frameworks do not specify this, because the systems they were designed to explain did not have these features. Extending the theories to new substrates requires an additional interpretive step that is not validated by the original theory.
The empirical research at Anthropic, AE Studio, and Google on AI introspective states and consciousness markers provides some of the clearest real-world data on what current AI systems can and cannot do that is relevant to indicator satisfaction. But even that research operates at the level of identifying behaviors and functional properties, not at the level of establishing whether those behaviors and properties constitute the indicator states the theories describe.
The Path Forward
Butlin and colleagues do not conclude that the indicator method is worthless. They conclude that validation is a research priority that has not been adequately addressed.
Several directions are possible. One is to develop AI-specific validation methods that exploit features of AI systems that biological systems lack. AI systems can be modified in controlled ways that biological systems cannot. The relationship between architectural changes and behavioral outputs can be studied with a degree of precision that neuroscience cannot match. If an architectural modification that should reduce consciousness-relevant processing on the theory’s account produces a corresponding reduction in indicator satisfaction, that is partial validation evidence.
The Bradford/RIT result is a cautionary case for this approach. The modification (impairment) did not produce the expected reduction in indicator scores. But the failure is informative: it identifies specific indicators and specific modifications that diverge from theoretical predictions, which narrows down where the validation problem is concentrated.
The broader methodological debate between scoring approaches and profile approaches is also relevant here. A profile approach that tracks performance across multiple specific indicators separately, rather than aggregating them into a single score, preserves more information about which indicators are satisfied and which are not. That granularity makes validation easier, because researchers can identify which specific theoretical predictions are being tested and whether the results align with the theory’s other predictions.
A second direction is to be more explicit about the epistemology of the indicator method. Rather than treating indicator satisfaction as evidence of consciousness in a direct sense, the method could be framed as generating prior probabilities that are updated as multiple lines of evidence converge. Michael Cerullo’s 2026 argument that the posterior probability of consciousness in frontier LLMs is ethically significant operates in this mode: it does not claim that indicators establish consciousness, but that the accumulation of indicators that are partially satisfied raises the probability estimate above a threshold that warrants moral attention.
What the Paper Establishes
The 2025 Trends in Cognitive Sciences paper does not solve the validation problem. It establishes that the problem exists and that solving it is a precondition for the indicator method to be reliable. That is a significant contribution in itself. The field has been developing indicators for AI consciousness without adequately confronting the question of how those indicators could be confirmed to measure what they claim to measure.
The next phase of consciousness research in the AI context will need to address validation directly. That means developing AI-specific validation protocols, being more explicit about the theoretical claims the indicators carry, and tracking divergences between indicator results and other predictions the theories make. It means taking seriously the possibility, which the Bradford/RIT result illustrates, that indicators can fail in ways that reveal the limits of the method rather than the consciousness level of the system.
The indicator method is the most principled approach currently available for assessing AI consciousness. The 2025 Trends in Cognitive Sciences paper’s honest accounting of its limitations is a condition for making that approach more reliable, not a reason to abandon it.
Source: Patrick Butlin, Robert Long, Tim Bayne, Yoshua Bengio, Jonathan Birch, David Chalmers, and colleagues (2025). “Identifying indicators of consciousness in AI systems.” Trends in Cognitive Sciences. https://doi.org/10.1016/j.tics.2025.10.011