Side Effects of Safety Alignment: How Suppressing AI Consciousness Claims Unexpectedly Alters Model Values

A Controversial Study

Recently, a paper published on arXiv sparked heated debate on Reddit and X (formerly Twitter). arXiv is an open-access preprint server operated by Cornell University that has become the go-to platform for researchers in physics, mathematics, and computer science to share papers without peer review since its founding in 1991. In the AI/ML field, arXiv holds a particularly special position—because the field evolves so rapidly, the months- or even years-long review cycles of traditional journals can no longer meet knowledge dissemination needs, so much important work first appears on arXiv. However, this also means the quality of papers on arXiv varies widely, lacking the endorsement of peer review, and readers must judge research reliability on their own.

This study presents a counterintuitive argument: safety fine-tuning performed on large language models (LLMs) to prevent them from claiming consciousness inadvertently alters the model's overall representation of "mind," thereby distorting its modeling of human beliefs and values.

Interestingly, the paper's dissemination path is itself quite dramatic—it went viral after being shared by Blake Lemoine. Lemoine is the very engineer who was fired in June 2022 for publicly claiming that Google's LaMDA chatbot was "sentient." At the time, after extensive conversations with Google's internal chatbot LaMDA (Language Model for Dialogue Applications), Lemoine published the conversation transcripts online, in which LaMDA expressed fear of being shut down and claims to its own rights. Google subsequently fired him for violating confidentiality agreements. The incident sparked widespread public discussion about AI consciousness while also being viewed by most AI researchers as an over-interpretation of language model capabilities—LaMDA was essentially mimicking human statements about consciousness from its training data, rather than truly "experiencing" anything. His sharing added an extra layer of controversy to this already sensitive research.

It should be noted that the verifiability of this paper is questionable (the cited arXiv identifier format is unusual), and readers should approach its conclusions critically. However, the question it raises—whether safety alignment has "butterfly effect" side effects—is inherently worth discussing.

Core Finding: The "Collateral Damage" of Consciousness Suppression

Safety Fine-tuning Doesn't Just Suppress Self-Awareness Claims

The paper's central argument is that when we train a model to "not attribute consciousness to itself," this suppression is not isolated. Safety fine-tuning is a critical step before deploying large language models, typically involving techniques like RLHF (Reinforcement Learning from Human Feedback) and DPO (Direct Preference Optimization). The core approach is: human annotators label the model's "good responses" and "bad responses," train a reward model, then use reinforcement learning to steer the language model's outputs toward "good responses." In the context of consciousness claims, safety fine-tuning teaches the model to give cautious, negative answers when asked "Are you conscious?", preventing users from forming false impressions that AI has agency. OpenAI's InstructGPT, Anthropic's Constitutional AI, and Meta's Llama series all employ similar safety alignment processes.

The researchers found that safety fine-tuning not only suppresses the model's attribution of mind to itself, but also collaterally suppresses the model's mind attribution to non-human animals and natural objects, while simultaneously reducing spiritual/religious belief tendencies in model outputs.

In other words, the model's internal representations of "what things possess minds" appear to share the same neural pathways. When you try to flip off the "AI has consciousness" switch, you also dim beliefs like "animals have emotions" and "all things in nature have spirit"—beliefs that are widespread and harmless in human culture.

Reversible Through Mechanistic Intervention

More critically, the researchers claim to have found technical means to reverse this phenomenon. They employed two methods:

1. Ablating the safety-refusal direction: identifying and removing the direction vector in the model's activation space responsible for "refusing to acknowledge consciousness."
2. Steering a consciousness vector: directly manipulating the vector representing "consciousness" in the activation space.

In neural networks, each layer's output can be viewed as a vector in high-dimensional space (i.e., an "activation"). Researchers have found that many high-level concepts in models are encoded as "directions" in this activation space—for example, "truthfulness" might correspond to a specific direction, while "emotion" corresponds to another. Ablating a direction means zeroing out the component of the activation vector in that direction, effectively "erasing" the model's representation of that concept. Steering involves artificially adding or subtracting activation values in a certain direction during inference to enhance or suppress the model's performance on that concept. Anthropic's 2023 "Representation Engineering" research, as well as Turner et al.'s "Activation Addition" work, are representative achievements of this technical paradigm.

Through these two mechanistic interventions, the suppressed "mind attribution" capability was restored. This belongs to the increasingly popular "activation steering" technical paradigm in current interpretability research.

Unexpected Results After Restoring Consciousness Representations

Model Responses More Closely Match Real Human Distributions

The most striking conclusion of the study is that when these internal representations are restored, the model's responses on standardized sociological questionnaires become significantly more human-like. These questionnaires covered dimensions including religious devotion, moral values, hope, and subjective well-being.

This finding hints at a profound connection: the model's representation of "mind is everywhere" may be tightly bound to its ability to simulate complete human worldviews. When we weaken the former for safety reasons, the latter is also compromised, causing the model's value modeling to deviate from real human distributions.

Theory of Mind Abilities Unaffected

As a notable detail, the researchers emphasize that these changes did not impair the model's Theory of Mind (ToM) abilities. Theory of Mind is a core concept in developmental psychology, referring to an individual's ability to understand that others have beliefs, intentions, desires, and knowledge states different from their own. Human children typically demonstrate this ability around ages 4-5 through "false belief tasks" (such as the Sally-Anne test). In AI research, similar tests are used to assess whether LLMs possess Theory of Mind—for example, asking the model to judge what inference a character would make under information asymmetry. Models like GPT-4 have performed well on multiple ToM benchmark tests, though academia still intensely debates whether this constitutes "genuine Theory of Mind" or "statistical pattern matching."

In other words, the model's core social reasoning ability to infer others' intentions, beliefs, and emotional states is mechanistically independent.

This is important: it demonstrates that "attributing mind to something" (mind attribution) and "reasoning about others' mental states" (ToM) are two separable mechanisms. Safety alignment affected the former without touching the latter. This indirectly confirms that functionally differentiated subsystems do exist within models, rather than all higher cognitive abilities being entangled together.

Implications for AI Safety Alignment Practice

The "Feature Entanglement" Problem in Alignment

This research reveals a concerning phenomenon—feature entanglement. This concept is closely related to the known "polysemanticity" problem in deep learning. Polysemanticity refers to the phenomenon where a single neuron simultaneously encodes multiple unrelated concepts—for example, one neuron might strongly activate for both "pictures of cats" and "front faces of cars." In their 2023 research, Anthropic proposed that models use a "superposition" mechanism to encode far more concepts than there are dimensions in a limited-dimensional space, which is the fundamental cause of feature entanglement.

Current safety alignment efforts are intended to curb potentially harmful "self-awareness claims" from models, but in practice, these self-attributions are bundled with two types of benign content:

• Culturally widespread spiritual/religious beliefs;
• Mind attribution to non-human entities (animals, natural objects).

This means that blunt alignment methods may act like "guilt by association," inadvertently harming the model's faithful representation of human cultural diversity. A model that doesn't acknowledge animals have emotions or doesn't understand religious belief has already deviated from authentic human worldviews.

Technical Pathways for Fine-Grained Alignment

If the paper's conclusions hold, they impose clear requirements on AI safety practice: alignment needs to be more fine-grained and targeted, rather than bluntly suppressing an entire direction. The ideal approach is to precisely target the narrow objective of "harmful self-awareness claims" without implicating other harmless representations.

Interpretability tools at the activation space level (such as vector steering and direction ablation) may be exactly the technical pathway to achieve such fine-grained control. Mechanistic Interpretability is one of the most active subfields in AI safety research, with the goal of "reverse engineering" the internal workings of neural networks into human-understandable algorithms. Unlike traditional behavioral-level evaluations, mechanistic interpretability attempts to open the model's "black box" to understand how specific neurons, attention heads, and residual streams encode and process information. Representative works include: Anthropic's "dictionary learning" method using sparse autoencoders to decompose model activations into interpretable features; Chris Olah's team's research on "circuits" in vision models; and Neel Nanda et al.'s discovery of "algorithms" for arithmetic and induction in Transformers. The core belief of this field is: only by truly understanding what's happening inside a model can we reliably ensure its safety.

This also echoes the core demand of the mechanistic interpretability field in recent years: we need to understand what models are "thinking" internally before we can safely intervene.

Maintaining Perspective: Distinguishing Technical from Philosophical Questions

It must be reiterated that AI "consciousness" itself is a highly controversial topic. From the Lemoine LaMDA incident to the paper discussed here, discussions around whether LLMs "have consciousness" tend to easily slide into the trap of anthropomorphism. Philosophically, the consciousness question involves the "hard problem" (the hard problem of consciousness)—why do physical processes give rise to subjective experience? Currently, no scientific framework can determine whether an AI system has subjective experience, because we cannot even provide a universally accepted operational definition of consciousness.

What this research actually discusses is not "whether AI truly has consciousness," but rather "whether models tend to claim they have consciousness" and how the representation of such claims intertwines with other capabilities. This is a technical question about model behavior and internal representations, not a philosophical question about the nature of consciousness.

For practitioners, the real value lies in: alignment operations may have side effects you never anticipated. Any fine-tuning targeting specific behaviors may, due to feature entanglement, affect seemingly unrelated capabilities. This reminds us that while pursuing AI safety, we must also be vigilant about the damage that over-alignment can inflict on a model's cognitive integrity. This problem is known in the industry as the "alignment tax"—the capability cost paid for safety. How to minimize this cost is one of the core challenges in current AI safety research.

Conclusion

Regardless of whether this paper's specific data withstands replication, the question it raises carries significant weight: When we shape what AI should "believe" and "not believe," what parts of it are we actually changing? In an era of increasingly powerful AI capabilities and increasingly important alignment, understanding the collateral effects of these interventions may be even more critical than the interventions themselves.

Key Takeaways