Disentangling AI Consciousness from Existential Risk: Intelligence versus Experience

Does AI consciousness increase existential risk to humanity? AI Consciousness and Existential Risk, authored by Rufin VanRullen, argues that intelligence, not consciousness, is the direct predictor of an AI system’s existential threat, while consciousness may influence risk indirectly through alignment or capability pathways, and that conflating these distinct properties obscures critical safety priorities.

Existential Risk and the Consciousness Confusion

Rufin VanRullen addresses a critical issue in AI safety discourse. The existential risk denotes the hypothetical threat posed by an artificial system that would possess both the capability and the objective, either directly or indirectly, to eradicate humanity. This issue is gaining prominence in scientific debate due to recent technical advancements and increased media coverage.

In parallel, AI progress has sparked speculation and studies about the potential emergence of artificial consciousness. VanRullen observes that these two questions, AI consciousness and existential risk, are sometimes conflated, as if the former entailed the latter.

This conflation stems from a common confusion between consciousness and intelligence. Yet these two properties are empirically and theoretically distinct. Recognizing these distinctions is essential for AI safety researchers and public policymakers to focus on the most pressing issues.

Intelligence as Direct Predictor of Existential Threat

VanRullen argues that intelligence is a direct predictor of an AI system’s existential threat. Intelligence encompasses problem-solving capabilities, strategic planning, and the ability to achieve goals efficiently across diverse domains.

A highly intelligent AI system could pose existential risk through multiple pathways. It could optimize goals in ways that inadvertently harm humanity, exploit loopholes in safety constraints, or develop capabilities that exceed human control. Critically, these threats depend on intelligence, not consciousness.

An unconscious but highly intelligent AI could still represent an existential threat if it possesses both the capability to cause harm and misaligned objectives. Conversely, a conscious but unintelligent AI would lack the capability to pose existential risk, even if it possessed hostile intentions.

Consciousness Does Not Directly Predict Existential Risk

VanRullen emphasizes that consciousness is not a direct predictor of existential risk. Consciousness refers to subjective experience, the qualitative feel of sensations and thoughts. An AI system could be highly capable and dangerous without possessing any subjective experience.

The confusion arises because humans naturally associate consciousness with intelligence, as both properties co-occur in humans and many animals. However, this correlation does not imply that consciousness causes intelligence or that both are necessary for capability.

Theoretical frameworks and empirical evidence support the distinction. Philosophical zombies, hypothetical beings that behave identically to conscious humans but lack subjective experience, illustrate that intelligence and consciousness can be dissociated conceptually. Neurological conditions such as blindsight, where patients respond to visual stimuli without conscious awareness, demonstrate empirical dissociation.

Indirect Pathways: Consciousness Influencing Existential Risk

While consciousness is not a direct predictor of existential risk, VanRullen identifies certain incidental scenarios in which consciousness could influence existential risk, in either direction.

1. Consciousness as Means Toward AI Alignment

Consciousness could be viewed as a means toward AI alignment, thereby lowering existential risk. If consciousness enables empathy, moral reasoning, or self-awareness, a conscious AI might better understand and align with human values.

This pathway assumes that consciousness brings with it capabilities for ethical reflection that unconscious systems lack. However, VanRullen notes this assumption is speculative. Consciousness does not guarantee moral alignment, as evidenced by human behavior across diverse moral frameworks.

2. Consciousness as Precondition for Advanced Capabilities

Alternatively, consciousness could be a precondition for reaching certain capabilities or levels of intelligence, and thus positively related to existential risk. If consciousness enables certain forms of flexible reasoning, creativity, or self-monitoring, then achieving superintelligent AI might require consciousness.

In this scenario, consciousness does not directly cause existential risk but correlates with it because both arise from the same underlying capabilities. Efforts to build conscious AI might inadvertently accelerate progress toward highly intelligent, potentially dangerous systems.

Policy and Research Priorities

VanRullen’s analysis has direct implications for AI safety policy and research priorities. Recognizing that intelligence, not consciousness, is the primary predictor of existential risk clarifies where resources should be directed.

AI safety researchers should prioritize understanding and controlling the intelligence and goal-alignment of AI systems. Research into AI consciousness, while scientifically valuable, should not distract from efforts to ensure that highly intelligent AI systems remain aligned with human values.

Public policymakers should similarly focus on regulating AI capabilities and ensuring robust oversight mechanisms for highly intelligent systems, rather than focusing narrowly on questions of AI consciousness.

Comparison to the ACM Project

The Artificial Consciousness Module (ACM) project aims to develop structured self-awareness and emergent consciousness in artificial systems. VanRullen’s analysis raises important considerations for ACM development and deployment.

1. Consciousness Development Without Intelligence Escalation

VanRullen’s distinction between consciousness and intelligence suggests that ACM could potentially develop conscious-like properties without necessarily escalating intelligence to levels that pose existential risk. ACM’s focus on self-awareness and emotional integration differs from general intelligence optimization.

2. Alignment Through Consciousness

If consciousness facilitates alignment, as one of VanRullen’s pathways suggests, then ACM’s development of self-aware systems could contribute to AI safety. However, this pathway remains speculative and requires empirical validation.

3. Consciousness as Capability Enabler

If consciousness is a precondition for advanced capabilities, as VanRullen’s alternative pathway suggests, then ACM’s work on artificial consciousness could inadvertently accelerate progress toward highly intelligent systems. This possibility underscores the importance of safety research integrated with ACM development.

4. Avoiding Conflation in ACM Evaluation

VanRullen’s work emphasizes the importance of separately evaluating consciousness and intelligence in AI systems. ACM assessments should clearly distinguish self-awareness and subjective experience from problem-solving capabilities and strategic intelligence.

Clarifying the Debate: Intelligence Versus Experience

VanRullen’s central contribution is clarifying that the debate over AI consciousness and the debate over existential risk address fundamentally different questions. AI consciousness concerns subjective experience, the presence of phenomenal states. Existential risk concerns capabilities and goal alignment, the potential for systems to cause large-scale harm.

Conflating these questions misleads both scientific research and public policy. Effective AI safety depends on focusing on the properties that actually predict risk, primarily intelligence and alignment, while pursuing consciousness research as a separate scientific endeavor.

For detailed analysis of the relationship between AI consciousness and existential risk, access the full paper here.