We have all, to a greater or lesser extent, experimented with ChatGPT or other Large Language Models. That is why we may not be surprised to learn that they write very well, but we may be surprised to learn that they also know how to adapt their arguments to the person in front of them in order to convince them. And not only is their ability to convince surprising, even more disturbing is their ability to do it better than a real person. This finding is especially relevant in a world where AI assistants are integrated into messaging, social media and customer service platforms. This research by Salvi et al. confirms with solid data a growing concern: that these technologies can be used to manipulate, misinform or polarise on a large scale. 

Although the study was conducted with US participants, the personalisation and persuasion mechanisms tested are extrapolable to contexts such as Spain, where there is also a strong digital presence, growing exposure to AI-generated content, and increasing social and political polarisation. This can be problematic because, as discussed in the article - unlike humans, who need time and effort to adapt their arguments to different audiences - GPT-4 can adapt its message instantly and on a large scale, giving it a disproportionate advantage in environments such as political campaigns, personalised marketing or social media conversations. This automated microtargeting capability opens up new possibilities for influencing public opinion, but also exacerbates the risk of covert manipulation. The authors of the study therefore recommend that platforms and regulators take steps to identify, monitor and, if necessary, limit the use of language models in sensitive persuasive contexts. Just as targeted advertising was once regulated, perhaps the time has come to think about actions to control algorithmic persuasion.

This research paper focuses on the ability of GPT-4 to debate and persuade in conversations on topics of different levels of controversy. The authors conclude, through a study of 900 users with diverse socio-demographic profiles (varying gender, age, ethnicity, education level and political affiliation), that GPT-4 exhibits abilities equal or superior to humans to persuade on the topics being debated. 

Additionally, the results suggest that if the AI model has access to personal information about the human with whom it converses, then it is able to leverage that knowledge to improve its persuasive skills.  

Although the research is conducted in a restricted environment (discussions limited to a certain time and structure) and with a biased sample population (mainly Americans and regular users of a certain experimentation platform), the practical implications could be important. For example, for the purposes of exploiting AIs to persuade the population for lawful purposes (safe driving or waste recycling) or unlawful purposes (manipulation of the citizenry for political purposes). 

This journey of AIs as ‘agents of persuasion’ will therefore need to be monitored in the short, medium and long term from an ethical and risk mitigation perspective.  

On the other hand, in a real-world environment, the feasibility of accessing certain personal data (e.g. ethnicity or political affiliation) may be questionable. To test these results and analyse their transferability to other settings, it would also be necessary to extend the experimentation with conversations in other languages (e.g. Spanish) and humans from other, more diverse geographical backgrounds.

As noted by the MIT AI Risk Repository, one of the risks identified in the domain-based taxonomy is “disinformation, surveillance and large-scale influence”. Any article that helps to confirm and understand these risks is welcome. Above all, if it is done transparently (the code and dataset are available) for academic validation and to potentially extend to other LLMs. Considering the flattery issues presented by one of the latest versions of chatGPT, which, when discovered by users, forced OpenAI to perform a rollback, this study is extremely relevant. Flattery, combined with persuasion, increases the risk mentioned above.

It should be noted that this study should be extended to other LLMs such as Llama (from META), Claude (from Anthropic), Gwen (Alibaba) or others to determine whether this phenomenon is common and to analyse the scenario where the LLM collects information about the user and whether in that scenario (which is the usual one) persuasion is as marked as in the context of the study.