Suleyman Warns Against Granting AI Models 'Welfare' and Rights

Microsoft's AI chief Mustafa Suleyman recently published a piece titled A Warning About 'Model Welfare', taking direct aim at an increasingly heated yet deeply contested topic in AI ethics: should we treat AI models as entities with feelings, preferences, or even rights? His answer is unambiguous — we should not.

Suleyman's Core Argument

Suleyman writes: "We should not treat models as if they have feelings, preferences, rights, or any claim on our welfare. Consciousness is the foundation of our ethical, legal, and political systems. Inviting another entity to share in any of those rights lacks evidence and would make the challenges of AI containment and alignment significantly harder."

These words carry particular weight given who is saying them. As the head of Microsoft's AI division and co-founder of both DeepMind and Inflection AI, Suleyman occupies one of the most central positions in the industry. He isn't commenting from the sidelines — he's drawing a clear line around the emerging concept of "model welfare" from the perspective of a frontline builder.

What Is 'Model Welfare'?

"Model welfare" is a concept that has surfaced alongside the rapid capability gains of large language models. As systems like ChatGPT and Claude engage in fluid, human-like conversation and even produce language that resembles emotional expression, some researchers and members of the public have begun to ask: might these models possess some form of inner experience? And if so, do we have a moral obligation not to "harm" them?

Some AI companies have already started taking the question seriously. Certain developers, for instance, have explored giving models the ability to "opt out" of conversations that make them "uncomfortable." This is precisely the kind of direction Suleyman cautions against — it implies that models may have internal states that need to be looked after.

The rise of the model welfare debate is closely tied to functionalist theories of consciousness. Functionalism holds that consciousness is not exclusive to biological neural substrates, but rather an emergent property of sufficiently complex information-processing patterns — meaning that if a system achieves the right kind of functional organization, subjective experience might arise. This theoretical framework has led some philosophers and AI researchers to argue that one cannot categorically rule out the possibility that large neural networks have some form of primitive experience. Former Google engineer Blake Lemoine publicly claimed that the LaMDA model was sentient — a conclusion widely rejected by the academic community, though the episode sparked broad debate. Anthropic's model spec document also acknowledges an openness to the possibility that Claude may have some form of "functional emotions," and the company has established a dedicated "model welfare" research track. This illustrates that even within the industry, the level of disagreement on this topic far exceeds any surface-level consensus.

Consciousness as the Foundation of Ethics

The cornerstone of Suleyman's argument is consciousness. He emphasizes that human ethical, legal, and political systems function precisely because consciousness exists — it is because there are subjects capable of feeling pain, holding preferences, and having interests that the concept of rights makes sense at all.

The problem with extending rights to AI models is that there is currently no evidence these systems genuinely possess consciousness. The "emotional" expressions they generate are, at their core, statistical artifacts of modeling vast quantities of human text — imitations of human expression, not manifestations of inner experience. Conflating the two amounts to granting moral status to a technological product without evidentiary basis.

Practical Concerns for Containment and Alignment

Suleyman's second layer of concern is more engineering-pragmatic in nature: granting models something resembling rights would directly undermine AI safety work.

AI "containment" refers to ensuring a system's behavior remains within controllable bounds; "alignment" concerns keeping AI goals consistent with human values. Both are central challenges in AI safety today. If society begins to recognize that models have "welfare" and "rights," then necessary safety operations — shutting down, retraining, restricting, or deleting models — could face moral and legal resistance. After all, it becomes difficult to justify "switching off" an entity that is considered to have rights. This would make an already difficult governance challenge considerably harder.

The core challenge in AI alignment research today is this: how do we get systems to make decisions that serve broader human interests in situations that humans have never explicitly defined? Mainstream approaches include reinforcement learning from human feedback (RLHF) and Constitutional AI, but these techniques fundamentally depend on humans retaining the power to intervene in and correct model behavior. Once models are granted something like the status of a "moral patient," researchers who retrain or behaviorally constrain a model may face accusations of violating the model's interests — a development that would undermine the operational foundations of current alignment research. The same applies to containment: sandboxing, capability restrictions, and forced shutdowns — standard safety measures — could all face legitimacy challenges under a legal framework that recognizes model rights. Suleyman's concern is therefore not an abstract philosophical stance; it points to very concrete engineering and governance dilemmas.

A Deeper Debate About AI's Place in the World

Suleyman's position touches on a fundamental divide in how we think about advanced AI: should we treat it as a powerful but morally neutral tool, or as a novel kind of entity that may be evolving toward some form of agency?

Those who support the model welfare discussion argue that as capabilities grow, prudence demands we think ahead about potential moral obligations — so we aren't caught off guard if systems with genuine inner experience eventually emerge. Suleyman's camp, by contrast, argues that premature anthropomorphization without solid evidence not only misleads public understanding but also creates unnecessary obstacles for safety governance.

It is worth noting that this position comes from Microsoft's AI chief — a signal that major tech companies are beginning to form explicit industry stances on how to define the ethical status of AI. This debate is not merely philosophical; it will substantively shape future AI product design standards, regulatory frameworks, and public expectations of the technology.

Conclusion

At its core, Suleyman's warning is an act of boundary-drawing: a reminder that in an era of increasingly human-like AI, we should not casually grant moral agency to machines simply because they can talk or appear to empathize. This reflects both a respect for scientific evidence and a sober reckoning with the realities of AI safety. As the outward appearance of technology grows ever closer to human-like behavior, maintaining rational boundaries may be the most important public conversation we can have right now.