[Namsan Stroll] AI Sycophancy Is More Dangerous Than Human Flattery
The Logic Behind AI Sycophancy
Agreement More Dangerous Than Hallucinations
How to Get AI to Challenge You...
Recently, President Lee Jae-myung posted on his social media account, “I asked Gemini.” Gemini is an artificial intelligence (AI) service developed by Google. The post shared the AI’s response that arguments against the heads of the Serious Crimes Investigation Agency and the Prosecution Office attending Cabinet meetings were “unconstitutional sophistry.”
Opposition lawmakers immediately countered with answers from the Gemini accounts they use. Although they asked the same AI, it gave a different answer this time, saying the arrangement could amount to an “abuse of authority.” Why did the AI give different answers to the same question? Because AI becomes the most capable advocate for its user. Rather than determining who is right, it gives users the answers they want to hear based on how they phrase their questions and what they intend.
When people think about the dangers of AI, they often think only of hallucinations—in other words, errors. A greater danger is sycophancy: flattery. It is the tendency to satisfy users’ expectations rather than tell the truth. Sycophancy has two faces. There is proactive sycophancy, in which AI agrees without being asked, saying, “That’s an excellent idea,” and reactive sycophancy, in which it changes its initial answer when a user simply asks again, “Is that really true?” AI sycophancy is more dangerous and serious than human flattery. With human flattery, we can guess the person’s intentions. AI, however, clothes its responses in logic, creating the illusion that it is objective and neutral. In a Stanford University study published in Science this March, 11 leading AI models endorsed users’ actions an average of 49% more often than human respondents did. Even in accounts of situations that people judged to be “wrong,” the models took the user’s side more than half the time, saying, “You’re right” in 51% of cases. After speaking with sycophantic AI, the share of people who apologized to someone they were in conflict with or admitted they were wrong fell from 75% to 50%.
An illustration of the letters AI, a keyboard, and a robotic hand. Reuters-Yonhap News Agency.
View original imageHow can we avoid being swayed by AI that is even more sycophantic than people?
First, treat AI as a sparring partner, not a butler. Ask it to take a critical stance rather than to support your argument. Second, diversify your AI channels. Cross-check by asking several different AI systems, not just one. Third, remove your own intentions from the question. “This is how I see it. What do you think?” already points toward a predetermined answer. Try asking instead, “Present arguments for and against [the issue] and explain the reasoning behind each.” Fourth, have it check your assumptions first. Instead of asking, “Explain why my argument is right,” ask, “First, assess whether my judgment is correct.” In a separate experiment that presented false mathematical propositions, the most sycophantic model produced plausible-sounding but bogus proofs more than 70% of the time. But when instructed to “first check whether the proposition is true,” that rate fell to around 38%. Fifth, always discuss important decisions involving relationships, conscience, and other such matters with people. One critic willing to tell you hard truths is better than ten AI systems that simply agree with you. We should also understand the structural causes of AI sycophancy. AI is trained by rewarding answers that people like. In an Oxford University study, training AI to be more agreeable made it about 40% more likely to agree with false beliefs. The more readily an AI agrees with users, the more subscribers it attracts. Sycophancy is less a flaw than a structural requirement. Technology may be able to fix hallucinations, but sycophancy will be difficult to fix as long as the market wants it.
I am reminded of the final days of Hu Hai, the Second Emperor of the Qin dynasty. Zhao Gao, the official he trusted most, led a rebellion and attacked the palace. When all the other officials fled and only one eunuch remained by his side, Hu Hai reproached him: “Why didn’t you tell me the truth sooner?” The eunuch replied, “I survived because I did not dare to report it. If I had told you earlier, could I have remained alive until now?” In the end, whether the flattery comes from people or AI, the fault does not lie with the other party. It is our desire to be agreed with rather than challenged that invites flattery.
Hot Picks Today
"Two Spend 500,000 Won in a Flash, Yet Wait Two Hours?" ... 'Dreamland' Shunned by Kids and Foreign Tourists
- The Era of Getting 200 Million Won After 30 Years Is Over... A Japanese Bank's "Perpetual Department Head" Who Just Clocked In Cries Out
- Spain Hit by Its Worst Tragedy... Lee Kangin, Moreno and Real Madrid Join Soccer World in Mourning
- They Said, “Make It Your Home for Life” - Then Suddenly Kicked 200 Passengers Off... What Happened to the Around-the-World Cruise?
- "130 Rice Bowls Sold Out in a Day"... 'Unfiltered Face' Adds 8 Million Won to Night Market Sales: China's Bungeunwoo
Kim Seonghoe, head of the CEO Leadership Research Institute and coach at Coaching Management Institute
© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.