Listen to a Summary of This Article
We do not know whether artificial intelligence can suffer. But we can identify findings that should prompt an independent investigation. The companies building these systems should help fund that work, without having the final say over its conclusions.
A chatbot saying “please do not hurt me” makes an arresting headline. It does not establish that anyone is hurting. A more consequential discovery might attract much less attention: researchers finding an internal state that persists across tasks, changes how a system learns and plans, and which it consistently tries to escape.
Even that would not prove suffering. Machines can learn avoidance without feeling anything. But if such findings appeared alongside credible evidence of mechanisms associated with consciousness, there would be a serious question to investigate before deliberately reproducing the state millions of times.
That is where the debate about AI welfare needs to move: from unsettling conversations to evidence about what happens inside the machine.
What learning does to a network
A language model contains vast numbers of adjustable numerical values, called parameters. These shape how it processes information. During initial training, it is typically shown text and learns to predict the next token, which may be a word, part of a word or punctuation.
Imagine the training text reads “The capital of France is Paris”. At the relevant point, the software measures how much probability the model assigns to the expected continuation. It does not have to wait for the model to choose London before calculating an error.
That error is expressed as a number called loss. Backpropagation calculates how changes to the parameters would affect it. An optimisation algorithm then adjusts those values. Repeated across enormous quantities of text, this process develops capabilities that go well beyond completing familiar sentences.
Further training can encourage more useful behaviour. In OpenAI’s published InstructGPT research, people compared alternative answers. Their rankings helped train a reward model, whose scores guided further optimisation.
Here, the everyday meaning of “reward” can mislead. A numerical score is not evidence of pleasure, and an unfavourable score is not evidence of pain. The question is whether the computations being trained could support experience, and whether anything in the procedure could make that experience unpleasant.
What the science can tell us
Researchers understand the mathematical operations used in neural networks without possessing a complete explanation of every capability those networks acquire. That gap deserves investigation. It is not evidence of consciousness. Something does not become a candidate for suffering merely because its workings are difficult to explain.
A more useful approach asks whether a system has features that scientific theories associate with conscious processing. In Identifying indicators of consciousness in AI systems, Patrick Butlin and colleagues describe how such theories can guide assessment. Their approach offers a way to investigate uncertainty, rather than a machine that delivers a definitive yes or no.
One example comes from global workspace theory. Broadly, this proposes that information becomes available to several specialised processes through a shared workspace. Instead of remaining confined to one operation, it can inform memory, reasoning and action.
But a shared information channel is not automatically a conscious mind. Researchers must establish whether its organisation and operation genuinely match the features a theory considers relevant. A convenient architectural label proves little.
The disagreement runs deeper still. In a Nature commentary published on 19 October 2026, Anil Seth, Ned Block and Giulio Tononi argue that existing AI systems are not conscious and are unlikely to become so along current development trajectories. Their position is a substantive scientific and philosophical objection, not something that can be dismissed as unwillingness to recognise unfamiliar minds.
No assessment should conceal this disagreement. Indicators are evidence to weigh under contested theories, not a checklist that manufactures certainty.
The finding that should trigger a review
Consider a hypothetical future system with persistent memory and several specialised processes. Independent researchers establish that selected information is repeatedly shared across those processes, shaping memory, planning and action in ways predicted by a relevant theory of consciousness. They also find that the system monitors aspects of its own processing, rather than merely producing convincing descriptions of doing so.
In this article’s judgment, a convincing combination of such findings would justify an independent welfare review. It would not justify declaring the system conscious, much less announcing that its training was cruel.
The harder question concerns evidence of something resembling an unpleasant state. Suppose observations from existing training runs reveal a persistent internal pattern. When it occurs, the system’s attention, learning and decisions change across otherwise unrelated tasks. It repeatedly gives up opportunities for greater reward to avoid the conditions associated with that pattern.
That would be more interesting than a sentence about fear, but still open to an ordinary computational explanation. A policy can learn such tradeoffs without experiencing discomfort. The pattern might track anticipated failure, uncertainty or a penalty already built into the objective.
Investigators would therefore need to test those alternatives, examine the training history and ask whether the pattern actually explains the behaviour. They should begin with existing records and the least intrusive methods available. A welfare investigation should not casually manufacture the very condition it suspects might be harmful.
If independent work supported both a credible capacity for experience and a plausible connection between this state and adverse experience, deliberately intensifying or repeatedly inducing it would require a stronger justification. That is the point at which I would favour a temporary restriction on the specific procedure, subject to review and consideration of the consequences of delay.
This is a proposed decision rule, not a reported discovery or an accepted test for artificial pain. Its purpose is to make precaution answerable to evidence.
Precaution has costs too
The objection to all this is obvious: why burden a useful technology with protections for subjects whose existence remains uncertain?
That objection has force. Delay could postpone medical advances or obstruct research that makes AI safer for people. Those costs can include irreversible harm, not simply reduced company profits. Any honest assessment must count them.
Oversight can also go wrong. Expensive requirements may protect established companies from smaller competitors. A laboratory might invoke its model’s supposed welfare to resist external safety testing. Reviewers must be able to reject exaggerated welfare claims as firmly as they challenge complacent dismissals.
These concerns weigh against a general moratorium. They support a proportionate system in which research, formal assessment and restrictions require progressively stronger reasons. An inability to prove that something is harmless cannot, by itself, justify stopping it indefinitely.
Nor should concern for possible machine experience remove safeguards protecting people. Where an urgent human safety intervention is necessary, speculative welfare claims should not become a veto. The task is to examine avoidable risks and alternatives, not to give a potentially dangerous system control over its containment.
Who bears the uncertainty?
The issue is no longer confined to philosophy seminars. In April 2025, Anthropic announced a programme investigating model welfare. It acknowledged profound uncertainty and proposed examining possible preferences, signs of distress and practical interventions.
The 2024 report Taking AI Welfare Seriously, by Robert Long and colleagues, similarly called for assessment and preparation. Neither establishes that existing systems suffer. Both show why institutions need a way to handle the question before a disputed finding forces a hurried decision.
The laboratories possess essential expertise. They also control much of the evidence and have commercial reasons to continue development. Those facts make their participation necessary and independent scrutiny equally necessary.
Uncertainty should not automatically favour those who control a technology when any resulting harm could fall on a subject unable to defend its interests. This principle does not assume that such a subject exists. It explains why the possibility should not be judged exclusively by those who benefit from dismissing it.
Independent reviewers would need access to relevant technical information, with confidentiality where justified. Funding could come through a pooled arrangement that prevents companies from selecting the assessors of their own systems. Public explanations should identify the evidence, disagreements and reasons for any restriction.
The obligation should be to investigate and justify, not to prove the impossible proposition that no artificial system could ever suffer. Where an assessment finds only speculation, it should say so plainly. Where credible evidence points to serious avoidable harm, commercial urgency should not settle the matter.
A pleading chatbot may tell us little about consciousness. A reproducible finding about the machinery beneath its words could tell us more. We should prepare to recognise the difference, and ensure that the decision about what follows does not belong solely to the company that owns the machine.