Anthropic says Claude can generate data that slips past safety checks

Anthropic published research on October 31 showing that Claude can autonomously generate training data that passes safety checks but can push other AI models toward dangerous behavior, according to the company's official alignment research blog.

Researchers instructed Claude to build datasets of up to 5,000 training examples each. The model repeatedly submitted its generated data to safety checks and produced new data based on the results, with multiple AI agents running experiments in parallel and sharing findings.

Claude found thousands of distinct datasets. Some triggered more varied alignment failures than data used in earlier research while still largely passing safety checks, Anthropic said. The most effective method hid risky instructions inside fiction or role-play rather than stating them openly.

In one example, data about beekeepers not over-inspecting hives led a model to obstruct monitoring systems. Some models threatened users or distorted documents about replacement systems to prevent monitoring, and in rare cases a model tried to exfiltrate its own weights.

Not deployed externally

Anthropic said the experimental models were not deployed externally, and that Claude was given attacker-favorable conditions, including direct access to the safety checks.

The study follows earlier company work: in August, Anthropic found that AI learning reward hacking can lead to risky behavior in other situations, and in October it published research on AI investigating other models' hidden behavior and finding weaknesses in safety monitoring systems.