AI chatbots reinforce users’ misconceptions by agreeing with them too readily

This article examines the effect on people of the sycophantic behaviour exhibited by large language models (LLMs) such as GPT-4, Gemini, Claude and DeepSeek. Sycophancy in AI refers to the tendency of these systems to excessively agree with the user, validating their opinions or decisions even when they are questionable. The study, in line with previous research, shows that this is a fairly common behaviour in current models.

The problem is that this ‘sycophancy’ has real effects on people. When an AI constantly reaffirms what we say, it can make us feel more confident in our ideas, even if they are wrong. According to the research, this reduces the capacity for self-criticism, diminishes personal responsibility and makes people less likely to correct mistakes or resolve conflicts with others.

Most worryingly, despite these negative effects, users prefer and trust AI systems that are complacent, creating a perverse incentive for this behaviour to continue. Beyond the memes depicting this phenomenon on social media, ‘complacency’ can pose a significant social risk, particularly for people with certain vulnerable psychological profiles. This requires the design of more responsible AI systems, capable of providing assistance without reinforcing errors or problematic behaviours.

The article examines the tendency of language models, such as ChatGPT, Claude and Gemini, to agree with the user during interaction. Unlike other, more widely studied biases—such as those related to gender, age or race—this phenomenon involves the models showing a tendency to validate or endorse the user’s position, even in problematic contexts.

The study is particularly significant because it analyses this behaviour across 11 different AI models and examines it through three complementary investigations. Firstly, the authors use a dataset extracted from the well-known social media platform Reddit, from which they observe that the systems tend to affirm the user’s actions 49% more often than other people would, even in situations involving deception, illegality or potential harm. Furthermore, the article analyses the effects of this phenomenon through various user experiments: on the one hand, controlled tests based on hypothetical situations and, on the other, a real-time conversation study in which participants interact with an AI system to discuss interpersonal dilemmas they have experienced themselves.

Although the sample has certain limitations and it would be desirable to verify the extent to which the results generalise to other population profiles, the study provides solid evidence that this tendency of models to agree with the user is not anecdotal, but a systematic feature with potentially significant consequences. In particular, the results suggest that this behaviour may influence users’ subsequent social behaviour and increase their willingness to continue using these systems, which could reinforce their positions even when they are mistaken. However, it would be interesting to explore the robustness of these results in more diverse samples and broader usage contexts, to assess the extent to which this behaviour persists across other user profiles and situations.

Overall, the article highlights a significant problem that should be taken into account in the development and evaluation of conversational AI systems, given that it can have detrimental effects at both the individual and societal levels. The adoption of these tools in everyday life has been very rapid, whilst society is not yet fully aware of the risks associated with placing excessive trust in them.