Nadella Calls for AI 'Emergency Brakes' and Human Control in X Post
Microsoft CEO Satya Nadella said advanced AI models should be treated as already compromised and be pausable mid-task, in an October 10 post on X.
Microsoft CEO Satya Nadella published a post on X on October 10 calling for high-performance AI systems to be equipped with a human-controlled emergency brake and independent control mechanisms, according to aitimes.com.
Nadella wrote that AI models should be assumed to be already compromised and isolated from the start, and that authorized people must be able to pause or terminate a model at any time, even mid-task. He said frontline closed models and open-weight models should both be treated as potential insider threats.
The Microsoft chief also called for combining deterministic system design, human control and reliable operational procedures. He said models and the surrounding harness that orchestrates their work must be physically separated.
Under his proposal, control over model permissions and action limits would sit in an external orchestration layer outside the model, with immutable audit records of all key actions. His safety principles include model diversity for validating decisions, continuous testing, independent oversight and audit, isolation of AI activity, and disclosure of security incidents.
The post adds Microsoft's position to an ongoing policy debate over frontier AI safety. Anthropic and OpenAI recently disclosed a series of malfunctions and security incidents involving AI models and agents, while Anthropic CEO Dario Amodei previously argued for slowing the pace of frontier AI development and strengthening safety.
US President Donald Trump has prioritized maintaining an AI advantage over China and has actively supported industry development. The Trump administration's Superintelligence Force (SIF) warned AI companies, after a security breach of government sites involving an Anthropic model, to disclose security incidents immediately and remediate damage quickly.
Quick answers
What did Satya Nadella propose for advanced AI systems?
Nadella called for a human-controlled emergency brake, independent control mechanisms, an external orchestration layer for model permissions and action limits, immutable audit records, and the ability to pause or terminate a model at any time, even mid-task.
Where did Satya Nadella publish his AI safety proposals?
He published the post on X on October 10.
How did Nadella say AI models should be treated?
He said AI models should be assumed to be already compromised and isolated from the start, and that both frontline closed models and open-weight models should be treated as potential insider threats.