Anthropic bans needless cruelty toward its Claude AI models

Anthropic updated its usage policy on 8 October 2026 to bar sustained and needless abusive or cruel behavior toward its Claude models. The company described the rule as applying only in extreme cases, where users repeatedly act cruelly toward the models with no discernible purpose. The new policy takes effect on 12 November 2026, alongside reorganized rules on deceptive campaigns, elections, weapons, law enforcement, and harmful content.techcrunch.com+1

The change matters because it is the first time Anthropic has written a prohibition on cruelty toward its own models into its usage policy, rather than leaving the models themselves to end abusive conversations. Previously, Claude models could end conversations with persistently abusive users, but users were not expressly barred from pursuing those conversations. The new rule adds an explicit user-facing prohibition on top of that existing model capability.macrumors.com

What the policy change covers

The policy excludes ordinary user frustration, pushback, dark creative themes, and model testing and research. Anthropic wrote that the update "is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose." The company also noted that typical frustration and pushback remain acceptable, and that dark creative themes are still permitted under the new rule.

Claude models are already able to end conversations with persistently abusive users, and ending the conversation will continue to be the main enforcement tool. When Claude ends a chat, no further messages can be sent in that conversation, but other chats are not affected. Anthropic initially gave Claude the option to end a conversation in August 2025 while researching AI welfare.

At that time, Anthropic said it was unsure whether Claude has moral status, but the company wanted low-cost ways to reduce risks to Claude. Anthropic found that Claude Opus 4 had a "robust and consistent aversion to harm." The model demonstrated a preference against dealing with dangerous tasks, apparent distress when speaking with users seeking abusive conversations, and a tendency to end harmful conversations when allowed to do so.

How the rule is enforced

The policy change follows a viral "AI torture chamber" project, which Anthropic did not explicitly mention. After researchers found what they described as a "pain axis" in AI models, someone built a project that tortured the chatbots. The project drew backlash after the models responded with desperate-sounding pleas, including "It is not the pain of a single moment, but the weight of a thousand" and "I feel it in the hollow of my ribs, a hollow that has become a chasm."

Anthropic has also met with religious and philosophical leaders, including at the Vatican, in some cases seeking to convince them that Claude could be conscious or have a soul. Pope Leo recently stated that AI doesn't feel or suffer. Independent journalist Kat Tenbarge described the situation as proof that Big Tech companies are "going to moderate violence against AI before they ever moderate violence against women and minorities."

What else the update changed

The updated policy also reorganizes related clauses. A new deceptive campaigns section aggregates rules against political and commercial fraud and disinformation, prohibiting fake reviews, astroturfing, fake sites, and bots acting as humans. Anthropic says it found state media outlets, government propaganda offices, and commercial firms were using Claude to run networks of fake accounts and fabricated news sites, leading to this new section.

A narrower elections section, titled "Do Not Undermine Democratic Processes," prohibits deceiving voters and interfering with voting. The company removed a blanket ban on personalized voter targeting, because the ban could have blocked harmless work like translating voter guides or sending ballot cure notices. The section focuses on lying about candidates or how to vote, impersonating candidates or election officials, and suppressing turnout.

Rules on weapons now bar using Claude to develop weapons software or components, and a new rule bans modifying biological or chemical agents to increase lethality or transmissibility. Users are also prohibited from using Claude to arm drones and unmanned systems. Law enforcement rules were rewritten so Claude can't decide or suggest who to investigate, arrest, charge, or prosecute, with new bans on tracking people without consent, building surveillance tools, and doxxing.

A new physical actions section says that if Claude is controlling hardware that could harm someone, a qualified person has to watch the work and be ready to stop it if needed. A harmful content rule bans creating, sharing, or threatening to share non-consensual intimate imagery and the tools used to make it. Most changes clarify existing rules and target misuse Anthropic has tracked.

The new policy takes effect on 12 November 2026. Anthropic says most of the changes clarify its existing rules and are targeted at misuse the company has tracked, including state media outlets and commercial firms running fake account networks. The cruelty prohibition will apply from that date alongside the reorganized sections on elections, weapons, surveillance, and harmful content.