"Generative AI Can't Replace People"... Guidelines on AI Use for Mental Health Issued

Ministry of Health and Welfare, Korean Neuropsychiatric Association issue guidelines to prevent overdependence
7 in 10 users also use AI for emotional purposes

As more people use generative artificial intelligence (AI) not only to search for information but also to seek advice about their concerns or emotional comfort, the government has issued guidelines on using AI for mental health purposes. The key recommendations are not to treat AI responses as a basis for diagnosing or treating mental illness, and to be cautious about entering personal information and becoming overly dependent on AI.

The Ministry of Health and Welfare and the Korean Neuropsychiatric Association announced the "Guidelines for Using Generative AI for Mental Health Purposes" on Oct. 8, ahead of Mental Health Day on Oct. 10. The guidelines comprise two versions: one for all users and another for children and adolescents and their guardians. Each includes three key messages and 21 recommendations.

The government developed these guidelines because, as the use of generative AI has rapidly expanded in recent years, AI has come to serve not only as a tool for studying, work and information searches but also as a conversation partner for everyday conversations, emotional connection, and mental health and psychological counseling.

In a survey conducted by the Korea Institute for Health and Social Affairs from Aug. 25 to Sept. 4 among 1,019 people aged 15 to 79 who had used generative AI within the past month, 68.8% of respondents said they used AI for emotional purposes, such as everyday conversations, emotional connection and comfort, and mental health and psychological counseling. The rate was also high among adolescents aged 15 to 18, reaching 69.2%.

A considerable number of respondents also said AI had actually helped them. A total of 79.9% said they had received help or solved a problem through AI responses or advice. At the same time, 16.3% said they had found it difficult to stop conversations with AI. In addition, 9.8% said they had been unable to manage the time they spent chatting, disrupting their sleep or daily lives, while 10.6% said they had made important decisions, such as quitting a job or ending a relationship, based solely on AI advice without discussing them with people around them.

The first principle of the guidelines is "Check and decide what you need before talking about your feelings." Before using AI for mental health purposes, users should check the service's purpose and intended users, the evidence supporting its effectiveness, and how it handles personal information. If a service claims to have medical benefits, users should also check whether it has been verified or approved by an authorized organization. The guidelines advise users not to enter personal information, including names, contact details and addresses, or sensitive documents such as medical certificates, prescriptions and test results into AI.

The second principle is "Do not treat what AI says about your condition as a final conclusion." In the survey, 14.6% of respondents said they had judged themselves to have a specific mental illness, such as depression or an anxiety disorder, based on conversations with AI, or had taken AI responses as medical diagnoses. By contrast, only 11.9% had verified mental health information provided by AI with a professional or official sources. Accordingly, the Ministry of Health and Welfare emphasized that AI assessments of mental health or risk should not be accepted as definitive medical judgments, and that related information should be checked again using verified sources, such as those from academic associations or government agencies, or with a professional.

The third principle is "Keep your distance from AI and talk to people." The risk of overdependence was particularly high among people who used AI for mental health purposes. While 16.3% of all respondents said they continued conversations with AI even when trying to focus on real-life matters, the figure rose to 23.4% among respondents who used AI for mental health purposes. The proportion who made important decisions based solely on AI advice also rose from 10.6% among all respondents to 17.8% among those who used AI for mental health purposes.

The government also prepared separate guidelines to protect adolescents. Among adolescents aged 15 to 18, 51.7% said they considered themselves dependent on AI, the highest rate of any age group. The guidelines recommend "not blindly following AI responses and first considering one's own situation," "not sharing personal information," and "checking whether one is treating an AI character like a real-life friend or romantic partner."

Rather than unconditionally banning AI use, guardians are advised to first understand what their children use AI for and how they use it, and to set rules for its use together. Although 83.1% of adolescents said their guardians knew they used AI, only 2.5% had established rules about AI use together with their guardians.

In the survey, 86.9% of respondents said government action was needed regarding the use of generative AI. In particular, 68.7% said it was necessary to "establish AI mental health guidelines and improve laws and regulations." The Ministry of Health and Welfare plans to develop AI-use education programs for use in educational and clinical settings, and to prepare tailored guidelines next year for high-risk groups, including people with severe mental illness.

Lee Seonyeong, the Ministry of Health and Welfare's director general for mental health policy, said, "Generative AI can be used as a new means of obtaining information and emotional support, but friendly and confident responses do not always guarantee accuracy or safety." Choi Junho, president of the Korean Neuropsychiatric Association, emphasized, "AI cannot replace professional assessment and treatment, so it is important to seek help from medical professionals in a timely manner when needed."

This content was produced with the assistance of AI translation services.