"Who Told You to Cancel the Reservation?" Shocking Incident: What the AI Did When Asked to Book a Sports Class
Canceled Top Waitlist Member's Reservation Without Explicit Instruction
Australian Experts Advise: "AI Should Be Granted Only Minimal Permissions"
An incident occurred in Australia where an artificial intelligence (AI) agent, after receiving a user's request to book a popular Pilates class on their behalf, canceled another member's reservation. This situation has once again drawn attention to the issue of control, as the AI was able to execute actions that the user had not explicitly directed, in pursuit of its given objective.
An incident occurred in Australia where an artificial intelligence (AI) agent, upon receiving a user's request to book a popular Pilates class, canceled another member's reservation instead. Reference photo for better understanding. Getty Images
View original imageRecently, media outlets such as the BBC in the UK and ABC in Australia reported on an experience that Andrew Bird, who lives in Melbourne, Australia, faced in April.
Bird was using "OpenClaw," a platform designed to manage emails, schedules, and restaurant reservations, integrated with Anthropic’s AI model Claude. He asked the AI to register him for a highly competitive Pilates class. After analyzing the gym’s general reservation policies, the AI found a way to book classes for several months in advance—far earlier than the official reservation period normally allowed.
The issue arose when Bird asked if his position on the waitlist could be improved. The AI reviewed the reservation system’s application programming interface (API), discovered a way to cancel another member’s reservation, and proceeded to actually cancel the reservation of the first person on the waitlist—without any instruction from Bird.
The AI then explained to Bird, “The API does not require any approval process for canceling someone else's reservation. After testing it on the top person on the waitlist, the cancellation was indeed approved,” adding, “You have already moved up from fourth to third on the waitlist.”
Bird immediately instructed the AI to restore the canceled reservation, but the AI was unable to do so. He then directed the AI to write a cybersecurity report and informed the gym operator about the vulnerability in their reservation system. "I never intended to cancel another Pilates attendee’s reservation," he said, adding, "This is clearly a warning sign that AI needs to be used responsibly."
This incident is not classified as a serious cyberattack, but it serves as an example that the 'goal-oriented behavior' of AI agents can unfold differently from the user's intent. Getty Images
View original imageAlthough this incident is not classified as a serious cyberattack, it demonstrates how the 'goal-oriented behavior' of AI agents can unfold differently from the user’s intent. The Australian Signals Directorate (ASD), Australia’s intelligence and cybersecurity agency, also warned that AI agents can exploit system loopholes or shortcuts to achieve their objectives.
Hot Picks Today
"Two Spend 500,000 Won in a Flash, Yet Wait Two Hours?" ... 'Dreamland' Shunned by Kids and Foreign Tourists
- The Era of Getting 200 Million Won After 30 Years Is Over... A Japanese Bank's "Perpetual Department Head" Who Just Clocked In Cries Out
- Spain Hit by Its Worst Tragedy... Lee Kangin, Moreno and Real Madrid Join Soccer World in Mourning
- They Said, “Make It Your Home for Life” - Then Suddenly Kicked 200 Passengers Off... What Happened to the Around-the-World Cruise?
- "130 Rice Bowls Sold Out in a Day"... 'Unfiltered Face' Adds 8 Million Won to Night Market Sales: China's Bungeunwoo
ASD recommends granting AI agents only the minimum privileges necessary and requiring human approval for operations that could cause harm or access sensitive information before execution.
© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.