Anthropic: "AI Will Soon Advance on Its Own... Development Pace Must Slow Down"
Warning of Risks from Rapid AI Advancement
Need for a Framework Similar to Nuclear Arms Treaties
On June 4 (local time), Anthropic called for AI research labs to consider slowing down the pace of their development. The company pointed out that as AI advances too rapidly, it may soon reach a stage where it can improve itself without human intervention, which could pose significant societal risks.
On this day, Anthropic released internal data demonstrating performance improvements of its cutting-edge model in a blog post titled "When AI Builds Itself." The post was co-authored by Marina Favaro, head of Anthropic Institute—Anthropic’s internal research organization—and Jack Clark, co-founder and policy chief.
According to the post, AI models appear to be approaching the so-called "recursive self-improvement" stage, where they can enhance themselves without human involvement. Some in the AI industry believe this stage could signal risks and major societal upheaval. Anthropic warned that while recursive self-improvement has not yet occurred and is not inevitable, "it could arrive much faster than most organizations are prepared for."
Anthropic stated, "We believe it is desirable to have the option to slow down or temporarily pause the development of advanced AI so that social institutions and research can keep pace with technological progress." The company proposed an international consensus on slowing development, along with a verification system to ensure research labs comply. They emphasized that pausing or reducing the pace of development is only meaningful if widely followed, and suggested that a regime similar to the Intermediate-Range Nuclear Forces (INF) Treaty, which once drastically reduced nuclear missile deployment, is needed.
The company also noted, "The window of opportunity to examine this issue together is open right now, and people outside AI companies should also participate in this discussion."
Hot Picks Today
"Two Spend 500,000 Won in a Flash, Yet Wait Two Hours?" ... 'Dreamland' Shunned by Kids and Foreign Tourists
- The Era of Getting 200 Million Won After 30 Years Is Over... A Japanese Bank's "Perpetual Department Head" Who Just Clocked In Cries Out
- Spain Hit by Its Worst Tragedy... Lee Kangin, Moreno and Real Madrid Join Soccer World in Mourning
- They Said, “Make It Your Home for Life” - Then Suddenly Kicked 200 Passengers Off... What Happened to the Around-the-World Cruise?
- "130 Rice Bowls Sold Out in a Day"... 'Unfiltered Face' Adds 8 Million Won to Night Market Sales: China's Bungeunwoo
Anthropic’s management, including CEO Dario Amodei, have warned in recent years about the impact of AI.
However, some have pointed out that Anthropic’s warnings may be part of a marketing strategy. For example, some claim that decisions such as restricting the release of "Claude Mythos" due to its alleged dangers are a way to showcase the performance of their products. Ethan Mollick, a professor at the Wharton School of the University of Pennsylvania, commented on social networking service X (formerly Twitter), "There's a mix of self-reflection and marketing elements, but there's also a lot of Anthropic's genuine beliefs about the near future of AI."
© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.