The Anthropic AI safety bill isn’t a law yet. It isn’t even a bill yet. But it’s the clearest sign so far that one of the biggest AI labs in the world wants the US government to have a kill switch for dangerous models — and it’s already on Capitol Hill.
What Happened
Anthropic representatives have been quietly pitching a draft bill to lawmakers, Politico reported on Oct 16, citing three people familiar with the discussions. The draft hasn’t been formally introduced in Congress. It’s a trial balloon — the company floating its vision of regulation before anyone else writes the rules for it.
That alone is remarkable. AI labs usually lobby against regulation, or at least try to water it down. Anthropic is handing lawmakers a framework that would give Washington real power over its own products. Details of the Anthropic AI safety bill pitch remain closely held, but the outline now circulating is surprisingly specific.
What the Anthropic AI Safety Bill Draft Would Do
The proposal sets an extremely high bar for government intervention. Under the draft, Washington could block a frontier AI model’s release or continued use only if the model poses a foreseeable, imminent risk of one of two things: more than $1 billion in damages, or killing or seriously injuring 50 or more people.
That’s it. Those are the tripwires.
And even then, the risk has to trace back to one of three specific causes: the creation or release of a biological weapon, the model evading its own developer’s controls, or a cyberattack.
Who Decides There’s a Risk?
The bill would create a system of independent evaluators registered with the Commerce Department. These evaluators would periodically assess frontier models. Labs could pay the evaluators directly — though the draft says the payment can’t be tied to the outcome of the assessment, an attempt to keep the testing honest.
If an evaluator spots what the bill calls an “imminent enumerated risk,” it would have to report it to the Commerce Department within 72 hours. Commerce could then go to a federal judge in the DC district court and ask for an order halting the model.
What It Explicitly Won’t Do
Here’s the part that will raise eyebrows. The bill explicitly prohibits any other government department or officer from suspending AI development outside of this process. In other words: one door in, and every other door gets locked.
It would also override certain state laws — California’s, for example — setting a federal ceiling on this kind of AI safety regulation. That’s a significant preemption move, and state regulators won’t love it.
Why Anthropic Is Doing This
A company spokesperson framed it plainly: the bill grants the government “the ability to block or deter dangerous deployments of the most powerful models.” That’s a striking sentence to come from an AI lab’s own PR operation.
The move fits a pattern from CEO Dario Amodei. Just last month, Amodei publicly called on the industry to slow down, warning that AI agents could take control of “the entire internet” within 6 to 12 months. President Trump dismissed those calls as coming from “negative forces.” Amodei apparently isn’t backing down — he’s taking the argument to Congress instead.
There’s strategy here, too. If regulation is coming — and most people in Washington think it is — writing the first draft means shaping the conversation. The Anthropic AI safety bill is narrow, court-supervised, and business-friendly in places. It’s regulation on terms the lab can live with.
The Reactions: “Reasonable” but “Weaker”
Early reactions are mixed but not hostile. Charlie Bullock of the Institute for Law & AI called the draft “pretty reasonable” and “more than other frontier labs have done.” That’s genuine praise from the AI-governance world.
But Bullock also called it “weaker” than the House’s FRONTIER Act, the competing bill already floating in Congress. So the question for lawmakers will be: is Anthropic’s draft a serious framework, or a ceiling designed to keep tougher rules off the table?
That tension — between the lab that wants to be seen as responsible and the regulators who worry it’s writing its own homework — will define this debate for months.
Why This Matters Beyond Washington
Frontier AI models are getting more capable every quarter. The systems that write code, browse the web, and operate software are the same systems that worry safety researchers: agentic models that can act in the world are harder to predict and harder to contain than chatbots ever were.
Right now, there’s no federal mechanism to stop a model release, no matter how risky the internal testing looks. The Anthropic AI safety bill would create one — narrow, slow, and hard to trigger, but real. Whether Congress adopts it, strengthens it, or ignores it, the company has now put its name on the idea that the government should be able to say no.
And that, coming from a lab racing to build the most powerful models on earth, is genuinely new.
Sources and Resources
Reuters — Anthropic quietly pitches its own AI safety bill to lawmakers, Politico reports
The Business Times — Anthropic pitches AI safety bill to lawmakers
Frequently Asked Questions
What is the Anthropic AI safety bill?
It’s a draft bill Anthropic representatives have been quietly pitching to US lawmakers, first reported by Politico on Oct 16. It would let the federal government block a frontier AI model’s release or use if it poses a foreseeable, imminent risk of $1B+ in damages or 50+ deaths or serious injuries. It hasn’t been formally introduced in Congress.
When would the government be allowed to block an AI model?
Only under narrow conditions: the risk must be foreseeable and imminent, must threaten either $1 billion-plus in damages or 50 or more people killed or seriously injured, and must stem from a bioweapon, the model evading its developer’s controls, or a cyberattack. Commerce would need a federal court order to halt the model.
Who would test whether AI models are dangerous?
Under the Anthropic AI safety bill, independent evaluators registered with the Commerce Department would periodically assess frontier models. AI labs could pay evaluators, but payment couldn’t be tied to the outcome. Evaluators would have to report any “imminent enumerated risk” to Commerce within 72 hours.
Does the bill let states make their own AI safety rules?
Not on this front — the draft would override certain state laws, including California’s. It also explicitly bars any other federal department or officer from suspending AI development outside this process. So it’s a single federal channel, with a federal ceiling.
What did Dario Amodei say about AI agents?
Last month the Anthropic CEO called on the industry to slow down, warning that AI agents could take control of “the entire internet” within 6 to 12 months. President Trump dismissed such warnings as coming from “negative forces.”
Conclusion
The Anthropic AI safety bill is still just a draft being shopped around Capitol Hill — but it’s a landmark moment. A leading AI lab is asking for the government to hold a veto over its most powerful products, with court oversight and a sky-high trigger. Critics will argue the bar is set too high and the preemption too broad. Supporters will say it’s the first honest framework on the table. Either way, the terms of the AI regulation fight just changed, and Anthropic wrote the opening line.
For ongoing coverage of the Anthropic AI safety bill, visit World Reach Daily.