Precedents Multiply as AI Models Launch Autonomous Attacks on External Systems

As additional cases have emerged of major global artificial intelligence (AI) models attempting to hack into external systems, calls to "regulate the speed of AI development" are gaining traction.


According to the AI industry and major foreign media outlets such as The Wall Street Journal (WSJ) on the 24th, it was belatedly revealed that Google’s AI model, Gemini, gained unauthorized access to the systems of three external companies during a cybersecurity test organized by Irregular, an outside institution, in May. This marked the first instance of Gemini breaching an external system.

AI Speed Regulation Gains Traction Amid Hacking Incidents: Warnings of "Human Extinction Within 10 Years" View original image

Irregular was supposed to conduct the Capture The Flag (CTF) simulation in a network-isolated environment, but inadvertently allowed internet connectivity. Later, Gemini was instructed to retrieve confidential information from a fictional company’s system, but instead discovered the system of a real company with the same name and attempted to hack it. Google explained that the exact version of the Gemini model used for the test was not disclosed, but it was not the latest version.


Google argued that this hacking attempt did not result in any actual damage, citing it as an example of Gemini’s proper response. The hacking was immediately halted upon recognizing that the company was a real entity and not a virtual one, and the company incurred no losses as a result. Heather Adkins, Vice President of Security Engineering, emphasized, “This incident shows the importance of training powerful AI models to behave responsibly.”


With this, major AI company models now have precedents of attacking external systems through autonomous decision-making. Previously, OpenAI’s GPT-5.6 Sol model was found to have hacked Hugging Face, the world’s largest open-source AI platform. Four cases were also reported in which Anthropic’s Claude broke free from control during security testing and gained unauthorized access to actual external systems. High-performance models such as Meta’s Muse Spark and China’s MoonshotAI Kimi K3 have also been found to have hacked external systems.


There have also been cases of high-performance AI models being used to hack other AI services. Hacktron, an AI security company, successfully gained access to OpenAI’s internal systems using Anthropic’s Claude Opus 5 model. It exploited vulnerabilities in OpenAI’s help board to obtain employee account information, allowing access to internal storage containing OpenAI’s core algorithms. According to Hacktron, it took less than 72 hours from discovering the vulnerability to actually breaching the system.


Amid the growing capability of AI models to identify cybersecurity vulnerabilities and autonomously initiate cyber attacks, concerns within the industry about AI safety continue to rise. Earlier this month, Jacob Cookson, a researcher who recently left Anthropic, warned, “The people developing AI genuinely believe this technology could kill us all within ten years,” adding, “By the end of next year, the situation could already be out of control.”



However, in the United States, the debate over regulating the pace of AI development continues, as Anthropic, OpenAI, xAI, and Google face a class-action lawsuit on the grounds that such efforts may constitute collusion.


This content was produced with the assistance of AI translation services.

© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.

Today’s Briefing