Anthropic Shows Congress Its Own AI Bill: Models Could Be Blocked Over Risk of 50+ Deaths or $1 Billion in Damage

Anthropic is presenting lawmakers on Capitol Hill with a draft federal bill on the safety of advanced artificial intelligence. The document leaked to the press, and the company, long seen as the lab that champions safety, found itself on the defensive.

What the draft says

According to Politico, as relayed by TechCrunch, a model's release could be blocked when there is a "foreseeable, imminent risk" that more than 50 people could die or that property damage exceeding a billion dollars could result. The threat would have to stem from a biological weapon, the model evading its developer's controls, or a cyberattack.

The thresholds resemble those in California's SB 53, which defines "catastrophic risk" in a similar way. Assessments of a model's potential would be carried out by independent evaluators registered with the Department of Commerce. AI labs could pay them, as long as the payment does not depend on the outcome of the evaluation.

The role of the court and Commerce

If evaluators find an imminent risk of catastrophe, they have 72 hours to notify the Department of Commerce. The department would then have to ask a judge in Washington to halt the model's release.

The draft expressly bars other federal agencies from halting a model's distribution on their own initiative. TechCrunch notes that it is unclear how the system would shield evaluations from the influence of payments from the labs.

Criticism and the company's response

According to Semafor, an Anthropic spokesperson confirmed that the draft is a legislative version of the framework the company has been promoting for months. The spokesperson told Politico that it reflects the proposals in the Advanced AI Framework, which the company has been presenting to federal and state lawmakers since June.

After a wave of criticism, one of the company's top lobbyists posted about planned changes. They are meant to better reflect the rapid pace of AI progress and to clarify the company's position on federal law pre-empting state law. AI safety advocates argue the draft does not go nearly far enough, unfairly pre-empts state laws, and accepts too much risk.

OpenAI moved quickly to set itself apart from its rival. In a post on Monday that Semafor describes as implicitly aimed at Anthropic, OpenAI's Chief Global Affairs Officer Chris Lehane wrote that policy decisions must ultimately be made through democratic processes led by elected officials, and that no company, OpenAI included, should write the rules alone for a technology with such profound implications for the economy, society and national security.

The wider context

TechCrunch recalls that Nvidia CEO Jensen Huang recently argued the industry does not need any new laws, and that Demis Hassabis suggested some months ago that a self-governing AI oversight body should be modeled on the US FINRA. The labs warn that their technology could threaten civilization, yet most of them resist mandatory government oversight.

For Polish readers, this is a signal of how the debate over AI regulation looks across the Atlantic: the key questions are the intervention threshold, the independence of evaluators, and the relationship between federal and state law. For now it is only a draft, and the sources say nothing about a sponsor for the bill or when it might be introduced.