Anthropic CEO Amodei doubles down on open-weight risk stance while insisting he never called for a ban

Anthropic CEO Dario Amodei is sticking to his view that capable open-weight models carry significant risks, but he's not against open weights in principle.

In a blog post, he stresses that "Anthropic has never advocated for a ban on open-weights models," adding that models "that don't have dangerous capabilities are a public good."

But Amodeiy says he has two "nightmare scenarios": authoritarian states, especially China, could build more powerful models than the US and use them to gain military superiority or surveillance capabilities. And powerful open models could be misused for cyber or bio attacks. AI labs know from their usage data that such attempts are already happening with closed models as well.

Open weights are riskier, he argues, because they lack safeguards and a released model can't be recalled. While open models also empower defenders, Amodei sees a strong imbalance on biological threats, where sufficiently capable models could weaponize viruses quickly while building defenses would take years even in the best case.

A study by the UK AI Security Institute found that open-weight models trail frontier models by a few months on cybersecurity. Amodei has warned about the capabilities of his own models before, warnings that helped prompt a US ban, so he's at least consistent in his arguments.

Amodei wants to slow China down and avoid worst-case scenarios

Amodei is calling for stricter chip export controls so China can't train more powerful models than the US without US chips. He wants action against industrial distillation, which China reportedly uses to partly sidestep the chip restrictions and act as a "free rider" on US AI success. Critics counter that cutting off US chip sales to China mainly fuels the growth of China's own chip industry and that distillation is a legitimate and useful technique.

Amodeli also wants mandatory safety tests for all powerful models, regardless of where they come from or how they're licensed. On that last point, Amodei sees a growing consensus, pointing to recent proposals that would exempt less powerful models from startups and research.

Amodei's comments follow reports that US authorities are weighing steps to make it harder for US companies to use Chinese open-weight models, along with a letter signed by many major tech companies opposing such moves.

Critics accuse Anthropic of talking down open-weight models to protect its own business from cheaper competition. OpenAI is also said to still be lobbying against open weights, even though the company signed the letter.