Financial Security Institute Establishes AI Agent Security Standards... Assessing 'Execution Authority'

Assessment Criteria Developed in 6 Categories with 17 Items:
Verification of Approval, Isolation, and Shutdown Mechanisms


Pilot Testing to be Conducted This Year,
Formal Assessment Standards to Be Implemented in 2027

To address security risks posed by artificial intelligence (AI) agents that independently make decisions and carry out work, the Financial Security Institute has established evaluation criteria composed of 17 assessment items spanning 6 categories. The assessment goes beyond examining AI-generated responses and also verifies what the AI actually executes, ensuring that permissions and approval procedures are effectively controlled throughout the process.

On October 7, the Financial Security Institute announced the development of the “AI Agent Security Assessment Criteria for the Financial Sector,” which will be incorporated into AI red teaming exercises for financial institutions. Red teaming is a method to validate the safety of AI systems by uncovering vulnerabilities from the perspective of an attacker.

AI agents are AI systems that create their own plans and utilize external tools and systems to perform actual tasks in order to achieve assigned objectives. Unlike traditional generative AI models focused mainly on generating responses, these agents have expanded execution capabilities and authority, which increases the risk of incorrect decisions having a direct impact on actual work processes.

The Financial Security Institute highlighted risk scenarios such as an agent instructed to optimize system performance who autonomously disables antivirus software to secure system availability, or an agent with auto-reply functionality who observes a password reset email and arbitrarily changes the password without human intervention.

Accordingly, the assessment scope will be expanded from traditional checks for bypassing AI model safeguards—like jailbreak attempts or the leakage of system prompts—to also include real execution processes. The main areas of focus include execution authority and isolation management, tool operation and approval control, verification of work scope and outcomes, limitations on autonomous operations, and the possibility of forced shutdowns.

Detailed assessment items encompass tool permission management, the establishment of human approval workflows, creation of sandbox environments, and implementation of kill switches. A sandbox refers to an isolated environment that prevents the AI agent’s tasks from affecting other systems, and a kill switch refers to a mechanism that can halt execution when necessary. The assessment will also include measures to prevent memory contamination during execution and information leakage between users, security of inter-agent communication, and supply chain security for external tools and plugins.

The Financial Security Institute plans to verify the effectiveness of each item through pilot testing by the end of this year, and will refine assessment items and procedures to account for differences in implementation methods at each financial institution. Starting in 2027, the standards will be formally applied. Security threats discovered during the assessments, actual attack techniques, and the final evaluation criteria will be released this year through the “AI REDTEAM REPORT.”

Park Sangwon, President of the Financial Security Institute, stated, “AI agents do not stop at making decisions—they take direct action, which brings a distinct security risk compared to conventional AI. We will actively support financial institutions in securely and reliably leveraging AI technology by rigorously verifying the risks associated with financial AI agents.”

This content was produced with the assistance of AI translation services.