Anthropic Urges to "Pause AI Development" ... What’s Happening?

"AI Could Soon Reach the Level of Self-Improvement"

"Global Community Should Have the Option to Pause Development"

U.S. artificial intelligence (AI) company Anthropic has called for a temporary slowdown in the global pace of AI competition. The appeal comes from concerns that, as AI advances too rapidly, it could soon reach the level of 'self-improvement'—where it enhances itself without human intervention.

On June 4 (local time), Anthropic released internal data via its blog, demonstrating how quickly its AI models are progressing. The company asserted that it would be "highly beneficial" if nations around the world could slow the speed of AI development.

Anthropic. Reuters Yonhap News

In a joint blog post, Anthropic's Head of Research, Marina Favaro, and co-founder, Jack Clark, predicted that AI could soon reach the stage of 'recursive self-improvement.' This refers to the point at which AI systems improve their own performance without any guidance or intervention from humans.

If AI is able to update itself directly, the pace of performance improvement could accelerate exponentially. Some experts have raised concerns that AI achieving recursive self-improvement could cause disruption across society as a whole.

In response, Anthropic emphasized that it would be desirable for the international community to have the option to slow down or temporarily halt the development of cutting-edge AI, so that social institutions and AI alignment research can keep pace with technological progress.

The company also suggested not only reaching an international consensus to slow the pace of AI model development, but also establishing verification mechanisms to ensure that competing firms are actually complying with such agreements.

While they noted that recursive self-improvement has not yet become a reality—and is not inevitable—they warned that "it could arrive faster than most institutions are preparing for."

Co-founder Clark has consistently warned about the potential risks of AI. He stated that if AI models enter the stage of recursive self-improvement, they could fundamentally change civilization itself, adding, "Such technology has never existed before, but I believe it may emerge within the next two years, or perhaps even sooner."

Anthropic plans to continue discussions with political, academic, and industry stakeholders over the coming years regarding recursive self-improvement and the establishment of verification systems.

This content was produced with the assistance of AI translation services.