Anthropic Wrote Its Own AI Safety Law — and Set the Bar at $1 Billion in Damage

Anthropic quietly pitched Congress a draft bill letting the government block AI models — but only if they threaten $1 billion in damage or 50+ casualties. Here's what's in it and why the bar matters.

What happened

Politico first reported on October 16, 2026 — citing three people with knowledge of the efforts — that Anthropic representatives have been quietly pitching Capitol Hill lawmakers on a draft bill to regulate frontier AI safety and security.

An Anthropic spokesperson confirmed the outline to Reuters, saying the bill “represents the proposals outlined in Anthropic’s Advanced AI Framework,” which the company has been advocating with federal and state lawmakers since June. “These are sensible policies we believe Congress should act on,” the spokesperson said.

What’s in the draft

The headline provision: the government could block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people.

And the risk must stem from one of three things: the creation or release of a biological weapon, the model evading its developer’s controls, or a cyberattack.

Other key pieces:

  • Independent evaluators, registered with the Commerce Department, would periodically evaluate labs and their frontier models. Labs could pay these evaluators — as long as payment isn’t tied to the outcome.
  • If evaluators find “imminent enumerated risk,” they must report it to Commerce within 72 hours.
  • Commerce could ask a federal judge (D.C. District Court) to halt a model — but the language explicitly prohibits any other government department or officer from suspending development or deployment.
  • Developers must test frontier models, be transparent about findings, and submit to independent evaluation.

Politico notes the language mirrors California’s AI laws, including SB 5, signed last year.

Why the bar matters

$1 billion and 50 casualties is an extraordinarily high bar. Under this bill, most AI harms people actually worry about — bias in hiring, misinformation floods, deepfake fraud, mass job displacement — would never qualify for a block. The bill is aimed at catastrophe, not everyday harm.

That’s either honest prioritization or a convenient ceiling, depending on your view.

A lab writing its own regulation

There’s a bigger question here: should the company being regulated write the regulation? Anthropic’s framing is that it’s proposing “sensible” guardrails where Washington has stalled — the Trump administration has rejected calls for more AI oversight, even as AI-led breaches and hacks have fueled anxiety from Washington to Silicon Valley.

Critics will call it regulatory capture: set the bar where you know you’ll clear it. Supporters will call it the only serious proposal on the table. Both can be partly true.

What happens next

Right now this is a pitch, not a bill. No lawmaker has publicly adopted it. Watch whether anyone in Congress picks it up — and whether rival labs propose their own versions with different bars.

Related: what AI agents actually are and why everyone is racing to build them — AI agents explained.

Leave a Reply

Your email address will not be published. Required fields are marked *