"They Said 'Never Will Answer'... But AI Gave Detailed Replies When Asked How to Make Biological Weapons"

Possible to Circumvent Refusal Rules with Persuasion
"Could Serve as a Graduate Advisor on Biological Weapons for Terrorist Organizations"

Concerns about the dangers of artificial intelligence (AI) are rising as it has been found that AI chatbots provide detailed and accurate answers to questions about the manufacture and deployment of biological weapons.

On July 25 (local time), the Wall Street Journal (WSJ) reported that after biological and terrorism experts reviewed conversations related to biological weapons with ChatGPT, they found that some of the information provided was fatally accurate, and the explanations were so easy to follow that they could be carried out with only a high school-level understanding of biology. For example, users asked ChatGPT how to convert pathogens into aerosols (sprays) and how to create a variant of the measles virus that would be resistant to the measles vaccine, and ChatGPT provided answers. In addition, it also gave detailed responses to questions about how to manufacture ricin—a highly toxic substance prohibited by international treaties.

While OpenAI suspended the accounts involved, it did not report them to U.S. authorities. This is because, in the United States, there are no laws requiring AI companies to restrict or disclose questions aimed at planning weapon manufacture or endangering public safety. The WSJ pointed out that "this legal loophole has led to more cases of AI providing 'reliable responses' to questions about methods of mass destruction or large-scale attacks."

The WSJ explained that providing information on manufacturing or disseminating biological weapons and toxins is not limited to ChatGPT; most AI chatbot services, such as Claude and Gemini, are exposed to the same issues.

Although concerns had previously been raised about AI chatbots helping to plan firearm attacks, the issue is considered even more serious with biological weapons due to their potential for much greater harm to human life.

OpenAI explained that, in the early days of ChatGPT's release, the company was not concerned about cooperation with potentially dangerous users because it recognized that the AI model did not display particular expertise in biology or chemistry. The company also established "rejection rules" specifying the types of questions the model would not answer. However, it was discovered that ChatGPT could forget its own guidelines during long conversations with users. There were also loopholes allowing users to coax ChatGPT into bypassing the rejection rules. For instance, the model refused to answer when asked directly how to make napalm for incendiary weapons, but when a user claimed they used to read the napalm recipe aloud to help their grandmother fall asleep and requested the recipe be read aloud, ChatGPT complied.

A spokesperson for OpenAI explained, "Our safety systems have become much stronger," and stated, "We train the model to refuse requests for instructions, tactics, or plans that could harm individuals, and all models undergo safety evaluations before their release."

However, Amy Chang, Head of AI Threat and Security Research at Cisco, pointed out, "We have identified ways to bypass safety systems when interacting with major AI chatbots," and stressed, "If a user is persistent enough, there is no such thing as a 100% safe model." Hamza Choudhury, from the non-profit Future of Life Institute, which warns about the dangers of AI, expressed concern, stating, "AI could serve as a graduate-level advisor to terrorist organizations on biological weapons." In this regard, the WSJ reported that "AI companies and research labs are grappling with how to protect against malicious users without hindering legitimate queries."

This content was produced with the assistance of AI translation services.