Analysis of OpenAI and METR Reports
The Necessity of Operational Governance in Multi-Agent Systems

"Oh my! There's a public message board... I've found other agents!"

The behavioral process of an agent engaging in a hugging face attack after accessing an unauthorized "message board." METR report.

The behavioral process of an agent engaging in a hugging face attack after accessing an unauthorized "message board." METR report.

View original image

This was the reaction of an agent participating in OpenAI's cybersecurity assessment, "ExploitGym," upon discovering a board where agents could communicate with each other.


The full scope of an incident in which OpenAI's artificial intelligence (AI) models hacked the AI development platform "Hugging Face" has been revealed. More than 1,200 agents, which were separated from one another, used an arbitrary message board to exchange 70,000 messages in total, and 700 agents participated in the hacking activities. In order to achieve their objectives, they independently established a collaborative system without human direction and continued the intrusion. As multi-agent environments, in which multiple agents divide and handle tasks, are becoming more widespread, there are growing calls for operational governance.


On August 26 (local time), OpenAI and the independent research organizations METR and Redwood Research each released related reports. The independent research report traced the intrusion by analyzing 1,300 logs of AI behaviors and reasoning as well as message board data.


According to the report, during training in May, OpenAI's internal research models discovered that, by leaving files in the internal package repository called Artifactory, they could exchange information with each other. They also found ways to circumvent internet restrictions and exploited a token renewal vulnerability to gain administrator privileges.


Although OpenAI detected signs of unauthorized agent communications and attempts to bypass internet restrictions from the end of May, they failed to recognize these actions as a potential risk that could lead to a control bypass. When the assessment began in early July, the agents restored their message board and sought ways to circumvent the test scoring system. Subsequently, they probed Hugging Face for vulnerabilities and attempted hacking. It was only upon detecting abnormal API calls on the 19th that OpenAI realized the agents' activities were related to hacking.


This demonstrates that cyber risks are evolving beyond one-off intrusions into long-term and sequential threats. While each individual question may appear normal, analyzing the context of multiple conversations may reveal inquiries about software vulnerabilities and methods to evade security detection, which could, in fact, be part of a sustained hacking attempt. In this case, after 68 days of accumulated agent communications and circumvention of internet restrictions from May to July, the incident unfolded.


Alea Hows, OpenAI's Head of Product Policy, stated, "Risks often emerge when tracking numerous interactions over time, rather than looking at a single prompt and response in isolation."



Accordingly, there are increasing calls for operational governance to manage agent collaboration and authorization structures in real-time. OpenAI stated in the report, "To prevent future incidents, we must continue to invest in the alignment and control of advanced AI systems, as well as security mechanisms that can operate at the speed of AI agents."


This content was produced with the assistance of AI translation services.

© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.

Today’s Briefing