Anthropic is calling on top artificial intelligence (AI) companies to consider pausing advanced AI development, highlighting concerns that AI capabilities may soon advance faster than society can manage safely. The company has noted rapid improvements in AI systems’ ability to autonomously complete complex tasks, warning that this could lead to “recursive self-improvement” — a scenario where AI systems can enhance their own capabilities with minimal human input.
Such advancements, Anthropic argues, present significant challenges for oversight, safety, and governance. The company is advocating for a temporary, industry-wide halt to provide governments, researchers, and society time to establish necessary safeguards and gain a better understanding of the implications posed by these increasingly powerful AI systems. The call for a pause comes amid heightened attention on Anthropic’s advanced AI model, Mythos, which has demonstrated the ability to identify vulnerabilities in software code, raising concerns about the potential misuse of such sophisticated AI tools.
Anthropic stresses that any slowdown in development would need to involve multiple leading AI developers, with clear guidelines on when the pause would start, how it would be monitored, and under what conditions development could resume. They argue that a unilateral pause by one company would be ineffective if competitors continued their advancements at the same rate.
In a bid to foster broader discussions around AI governance, Anthropic’s research division plans to engage with policymakers, researchers, civil society organizations, and other AI companies to explore the risks associated with increasingly autonomous systems. This appeal for a coordinated pause comes as governments around the world continue to assess regulatory approaches for artificial intelligence, while major technology firms race to develop more advanced AI models.