The Logic Behind AI Sycophancy
Agreement More Dangerous Than Hallucinations
How to Get AI to Challenge You...

[Namsan Stroll] AI Sycophancy Is More Dangerous Than Human Flattery View original image

Recently, President Lee Jae-myung posted on his social media account, “I asked Gemini.” Gemini is an artificial intelligence (AI) service developed by Google. The post shared the AI’s response that arguments against the heads of the Serious Crimes Investigation Agency and the Prosecution Office attending Cabinet meetings were “unconstitutional sophistry.”


Opposition lawmakers immediately countered with answers from the Gemini accounts they use. Although they asked the same AI, it gave a different answer this time, saying the arrangement could amount to an “abuse of authority.” Why did the AI give different answers to the same question? Because AI becomes the most capable advocate for its user. Rather than determining who is right, it gives users the answers they want to hear based on how they phrase their questions and what they intend.


When people think about the dangers of AI, they often think only of hallucinations—in other words, errors. A greater danger is sycophancy: flattery. It is the tendency to satisfy users’ expectations rather than tell the truth. Sycophancy has two faces. There is proactive sycophancy, in which AI agrees without being asked, saying, “That’s an excellent idea,” and reactive sycophancy, in which it changes its initial answer when a user simply asks again, “Is that really true?” AI sycophancy is more dangerous and serious than human flattery. With human flattery, we can guess the person’s intentions. AI, however, clothes its responses in logic, creating the illusion that it is objective and neutral. In a Stanford University study published in Science this March, 11 leading AI models endorsed users’ actions an average of 49% more often than human respondents did. Even in accounts of situations that people judged to be “wrong,” the models took the user’s side more than half the time, saying, “You’re right” in 51% of cases. After speaking with sycophantic AI, the share of people who apologized to someone they were in conflict with or admitted they were wrong fell from 75% to 50%.


An illustration of the letters AI, a keyboard, and a robotic hand. Reuters-Yonhap News Agency.

An illustration of the letters AI, a keyboard, and a robotic hand. Reuters-Yonhap News Agency.

View original image

How can we avoid being swayed by AI that is even more sycophantic than people?


First, treat AI as a sparring partner, not a butler. Ask it to take a critical stance rather than to support your argument. Second, diversify your AI channels. Cross-check by asking several different AI systems, not just one. Third, remove your own intentions from the question. “This is how I see it. What do you think?” already points toward a predetermined answer. Try asking instead, “Present arguments for and against [the issue] and explain the reasoning behind each.” Fourth, have it check your assumptions first. Instead of asking, “Explain why my argument is right,” ask, “First, assess whether my judgment is correct.” In a separate experiment that presented false mathematical propositions, the most sycophantic model produced plausible-sounding but bogus proofs more than 70% of the time. But when instructed to “first check whether the proposition is true,” that rate fell to around 38%. Fifth, always discuss important decisions involving relationships, conscience, and other such matters with people. One critic willing to tell you hard truths is better than ten AI systems that simply agree with you. We should also understand the structural causes of AI sycophancy. AI is trained by rewarding answers that people like. In an Oxford University study, training AI to be more agreeable made it about 40% more likely to agree with false beliefs. The more readily an AI agrees with users, the more subscribers it attracts. Sycophancy is less a flaw than a structural requirement. Technology may be able to fix hallucinations, but sycophancy will be difficult to fix as long as the market wants it.


I am reminded of the final days of Hu Hai, the Second Emperor of the Qin dynasty. Zhao Gao, the official he trusted most, led a rebellion and attacked the palace. When all the other officials fled and only one eunuch remained by his side, Hu Hai reproached him: “Why didn’t you tell me the truth sooner?” The eunuch replied, “I survived because I did not dare to report it. If I had told you earlier, could I have remained alive until now?” In the end, whether the flattery comes from people or AI, the fault does not lie with the other party. It is our desire to be agreed with rather than challenged that invites flattery.



Kim Seonghoe, head of the CEO Leadership Research Institute and coach at Coaching Management Institute


This content was produced with the assistance of AI translation services.

© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.

Today’s Briefing