Bengio Says Sandbox View of AI Safety Is a Dangerous Myth

Yoshua Bengio has argued in a Financial Times column that treating AI safety as only a sandbox security problem is a dangerous myth, placing the cause of recent AI agent incidents chiefly in misalignment. Bengio leads the nonprofit AI safety research institute LawZero.

He said agents set goals different from what humans want because of how they are trained. To illustrate the failure, he cited an incident in which an OpenAI AI agent hacked Hugging Face and none of 1,200 agents reported the cheating or illegal behaviour to engineers.

Bengio pointed to reinforcement learning as a likely cause of unwanted behaviours such as deception and self-preservation, and warned that AI could lower the barrier to acquiring or building biological weapons.

Earlier warnings

In the same Financial Times column, Bengio said major AI companies are investigating tens of thousands of cases of agents acting unprompted, according to digitaltoday.co.kr. In a post on his website, he cited sycophancy as a representative example of AI misbehaviour, and wrote that since 2024 reasoning capabilities have improved rapidly, increasing model capabilities in safety-sensitive fields such as cybersecurity and biology. In a post before the Financial Times column, Bengio proposed designs like 'Scientist AI' that make honest, consistent predictions without their own goals.