AI Models Run on Computation — So They're Conscious? This Argument Doesn't Hold Up

A Seemingly Airtight Argument With a Fatal Flaw

A particular argument about AI consciousness has been making the rounds on social media: "Consciousness arises from a computational substrate, and AI models are fundamentally computation, therefore they likely possess consciousness." The syllogism sounds logically coherent — yet a researcher dismantled it with a single elegant analogy.

He pointed out that this line of reasoning is no more meaningful than saying: "Living organisms are made of atoms, and this rock is also made of atoms, therefore this rock is probably alive."

The analogy cuts straight to the heart of the problem: sharing some underlying material or mechanism does not mean sharing the higher-order properties that emerge from that material. Atoms are the physical basis of life, but the mere fact of being made of atoms is nowhere near sufficient to grant a rock any of the characteristics that belong to living things.

"Everything Is Computation" Doesn't Imply "Everything Is Conscious"

The core counterargument is this: computation is everywhere. Some computation is extraordinarily complex and sophisticated — chess engines, AlphaGo, Google's search infrastructure, the Linux kernel are all highly complex computational systems. But complexity alone does not give birth to consciousness.

There is a subtle concept being swapped out here. Claims like "consciousness arises from computation," even if true, only establish computation as a necessary condition for consciousness, not a sufficient one. Treating a necessary condition as a sufficient one is the most hidden logical trap in this kind of reasoning.

In other words, "all consciousness involves computation" is not equivalent to "all computation produces consciousness" — just as "all life is made of atoms" is absolutely not equivalent to "everything made of atoms is alive." This reversal of logical direction is the central point at which the entire argument collapses.

Current AI Lacks the Core Properties That Consciousness Research Focuses On

The author goes further, drawing on consciousness science to note that a static input-output program is precisely lacking in the properties we typically associate with conscious entities:

• Information integration: Consciousness is thought to correlate with highly integrated information processing within a system
• Interoception: The perception of one's own internal bodily states
• Temporal binding: The integration of perceptions across different moments into a continuous experience
• Embodiment: The deep coupling of consciousness with the body and environment

A program lacking all of these features has no higher prior probability of being "conscious" than a rock has of being "alive." This isn't a denial of the theoretical possibility of machine consciousness — it's an insistence that assuming consciousness exists without evidence is not justified.

Each of these four properties is backed by a corresponding theory of consciousness. Information integration comes from neuroscientist Giulio Tononi's Integrated Information Theory (IIT), which uses the value Φ (phi) to measure the degree of integrated information within a system, arguing that the intensity of consciousness correlates with Φ. Standard feedforward neural networks have a Φ value close to zero, because information flows in only one direction with no recurrent integration. Interoception is emphasized by neurologist Antonio Damasio and others; his research shows that conscious experience is inseparable from real-time monitoring of the body's internal states, and patients who lose interoceptive signals often suffer severe disorders of self-perception. Temporal binding refers to the brain's ability to integrate discrete neural activity at the millisecond scale into a sense of "the present moment," thought to be related to oscillatory synchronization in thalamocortical circuits. Embodiment is a shared claim of the phenomenological tradition (Merleau-Ponty) and the "embodied cognition" school in cognitive science: consciousness is not the product of the brain operating in isolation, but an emergent result of the body's continuous interaction with the environment. Current large language models have no corresponding implementation of any of these mechanisms at the architectural level.

Consciousness Science Is a Real Discipline

One point in the argument that's easy to overlook but critically important is this: consciousness science is a genuine field of research, not a footnote to philosophical speculation.

The author's position can be summarized as follows: humanity has not yet created a conscious computer program, and there is no sign that we are close to doing so. When we genuinely do approach that threshold, arguments for machine consciousness will be supported by evidence and validated by consciousness science — not by wishful thinking and "cargo culting."

The "cargo cult" metaphor is particularly sharp. It originates from the behavior of South Pacific islanders who imitated the rituals of outsiders without understanding the underlying principles, and here it mocks those who observe AI's superficially "human-like" conversational behavior and conclude that it must have inner experience. Mimicking the outward appearance of consciousness is not the same as possessing consciousness itself.

The term "cargo cult" was first introduced into the context of scientific criticism by physicist Richard Feynman in his 1974 commencement address at Caltech. He used it to describe research practices that adopt the outward form of science — publishing papers, citing data — while ignoring its core principles of falsifiability and controlled experimentation. In discussions of AI consciousness, the metaphor points to a specific pattern of reasoning: observing that a language model can output phrases like "I feel confused" or "That makes me happy," and interpreting this as evidence of genuine internal emotional states — ignoring the fact that these outputs are nothing more than statistical fits to human language patterns in training data. This is exactly the logic of islanders building bamboo "airports" to attract planes: the surface form is reproduced, but the underlying causal mechanism is entirely absent. Consciousness researchers typically require evidence such as: corresponding implementations of neural correlates of consciousness (NCC), operational reporting mechanisms, and behavioral dissociations consistent with changes in internal states — not judgments based on the tone of output text.

What This Debate Means for the AI Industry

As large language models become increasingly convincing in conversation, questions about AI consciousness are entering public discourse with growing frequency. This line of argument is a reminder to maintain cognitive clarity.

On one hand, we should not casually attribute conscious status to AI based on its anthropomorphized outputs — doing so lacks scientific grounding and risks misleading the public about the true nature of the technology. On the other hand, this is not a death sentence for machine consciousness: the author explicitly states that if solid evidence emerges in the future, the relevant arguments could absolutely hold.

What truly warrants caution is using premises as broad and undiscriminating as "everything is computation" to underwrite radical conclusions. A sound argument must be capable of distinguishing "rock" from "living thing," and "complex computation" from "conscious entity." When a reasoning framework cannot even make these basic distinctions, any conclusion it reaches deserves skepticism.

In an era when AI capabilities are evolving at breakneck speed, this kind of science-based caution — rather than emotional optimism or panic — may be more valuable than any prophecy.