"Humanity Could Become Extinct After Losing Control of AI," Warns Former Anthropic Researcher

Former Anthropic Researcher Jacob Coxon
Testifies at New York City Council AI Hearing

A former researcher at Anthropic, a U.S.-based developer of generative artificial intelligence (AI) services, has issued a warning about the dangers of the ongoing race to develop AI. He argued that humanity could lose control over AI, leading to possible extinction.

Jacob Coxon speaking at the New York City Council hearing on the 5th (local time). Photo by Yonhap News Agency, Reuters

On the 5th (local time), Bloomberg and CNBC reported that former Anthropic researcher Jacob Coxon stated during a New York City Council hearing on AI, "We still do not know how to control any AI system." Coxon, who left Anthropic last month, previously criticized Anthropic and OpenAI for "betting with our lives."

This time, Coxon was called to testify at the request of New York City Council Speaker Julie Menin, who is promoting a package of AI regulatory bills. He warned, "On our current trajectory, the likelihood that humanity will lose control of AI is greater than the likelihood that it won't," adding, "This could ultimately lead to human extinction."

Coxon pointed out that the methods of development in the AI industry are "extremely reckless." He continued, "The approach of 'move fast and break things,' a longtime Silicon Valley motto, might work for a photo-sharing app but not for building the most powerful technology in history."

Regarding the 'AI kill switch' emergency shutdown device under consideration by the City Council, he said it might help in the short-term but would not be sufficient as a long-term safety measure.

Alex Turner, a former Google DeepMind researcher, also participated via video conference to highlight the dangers of development competition. He noted, "China is not the only potential adversary," and explained, "There is a quite high probability that we are competing domestically to create our own adversary, that is, an unaligned AI." He warned, "This could one day become more powerful than China."

This content was produced with the assistance of AI translation services.