Anthropic CEO Rejects Open-Weights Ban, Pushes Alternative Controls

Dario Amodei, CEO of Anthropic, said his company has never advocated for banning open-weights AI models, responding to recent accusations as debate over Chinese open-weights models continues. In a post released Wednesday, Amodei described open-weights models without dangerous capabilities as “a public good” and argued that protectionist bans would not address his most serious national security concerns.

Amodei identified two “nightmare scenarios.” The first is that authoritarian governments—most prominently China—build AI models more powerful than those developed in the US, using them for permanent military advantage or deep domestic surveillance and repression. The second is that powerful models are misused for cyberattacks or biological attacks, with open-weights models presenting a higher risk because guardrails and usage monitoring are harder to enforce and weights cannot be withdrawn once released.

On the second scenario, Amodei explicitly rejected a US-focused ban: “Banning the use of these models by US businesses does nothing to address this risk, because bad actors are unlikely to be legitimate US businesses.” He acknowledged such a ban would protect US AI companies from competition but stated that was never his goal.

Instead, Amodei outlined three measures he and Anthropic have consistently supported.

First, the US should not sell powerful chips or chipmaking equipment to China and should crack down on smuggling and workarounds. Because of scaling laws, he argued, China cannot build models more powerful than those in the US without access to US chips. Second, the US should target industrial-scale distillation operations, which allow China to build better models than its chip count would ordinarily permit, partially evading chip bans. Distillation does not let China match or exceed the US frontier, Amodei said, but can bring it within months. Third, all sufficiently capable models—open and closed—should undergo mandatory safety testing for cyber, biological, and alignment risks before release. He noted that the Trump administration has moved in this direction and that industry proposals have emerged that would apply testing to the most capable models regardless of origin or openness.

Amodei largely agreed with an open letter supporting open-weights models, citing benefits like expanded access and competition. But he disagreed with the letter’s claim that open-weights models necessarily make it easier to develop safeguards or that broad access helps defenders more than attackers. He raised the example of biology, where “sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials,” while defense remains a multi-year task. “Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance,” he wrote.

The post raises an unresolved question: whether global safety testing is feasible when one of the principal actors—China—would need to cooperate. Amodei acknowledged that challenge but expressed cautious optimism, noting that limited cooperation on preventing AI biological weapons may be possible because it aligns with China’s own interests. Whether that optimism is warranted remains to be seen.