Anthropic Drafts Bill to Block Risky AI Model Releases
The draft, reported by Politico, would let a judge halt a model's release if it poses a foreseeable risk of 50 deaths or $1 billion in damage.
Anthropic is circulating a draft AI safety bill that would allow a model's release to be blocked when there is a foreseeable, imminent risk that more than 50 people could die or more than $1 billion in property damage could occur, according to Politico. An Anthropic spokesperson confirmed the draft bill to the outlet.
The draft limits such catastrophic risk to three cases: the creation or release of a biological weapon, the model evading its developer's controls, or a cyberattack. Independent evaluators registered with the U.S. Commerce Department would assess a model's potential for disaster. AI labs would be allowed to pay those evaluators, provided the payment is not contingent on the results.
Under the proposal, evaluators would have 72 hours to notify the Commerce Department after finding an imminent catastrophic risk. The department would then have to ask a D.C. judge to halt the model's release. The bill expressly forbids any federal agency from stopping a model's distribution on its own discretion.
Earlier proposals on AI oversight
Anthropic has been advocating its Advanced AI Framework proposals with federal and state lawmakers since June, according to a spokesperson. California recently passed Senate Bill 53 into law, which defines similar thresholds as catastrophic risk.
Other industry figures have taken different positions on regulation. Some months ago, DeepMind chairman Demis Hassabis said a self-governing AI regulatory body should mimic the financial industry's FINRA. Recently, Nvidia CEO Jensen Huang argued that the AI industry did not need any new laws.
Quick answers
What does Anthropic's draft AI safety bill propose?
It would let a model's release be blocked when there is a foreseeable, imminent risk of more than 50 deaths or more than $1 billion in property damage from a biological weapon, the model evading its developer's controls, or a cyberattack.
Who would assess AI models under the draft?
Independent evaluators registered with the U.S. Commerce Department, who would have 72 hours to notify the department of an imminent catastrophic risk.
Has the bill been introduced in Congress?
No. Anthropic is circulating the draft in the hope that Congress will consider adopting it, according to Politico.