"We have built a tool that may be optimised for convincing people of things that are not necessarily true." Soumitra Dutta, former dean of the Saïd Business School at the University of Oxford and AI scholar, considers misinformation a major risk of artificial intelligence.

Dutta, who is cofounder of AI startups NexiVerify and Caasaa, is not the first expert to air this concern. Over the past couple of years or so, researchers have conducted experiments to demonstrate the argument. University of Washington professors Carl Bergstrom and Jevin West have famously used the phrase "superhuman ability to bullshit" to describe LLMs.

A 2025 Nature Human Behaviour article by Francesco Salvi‚ Manoel Horta Ribeiro‚ Riccardo Gallotti‚ and Robert West furnishes strong evidence: GPT-4 and human debaters debated human participants and those who debated a personalized GPT-4 changed their opinion more easily compared to those who debated other humans․ (However‚ it’s worth adding that without personal knowledge‚ GPT-4 was not more persuasive than a human․)

Giving the model facts about the person it was debating gave it an even bigger advantage. It wasn't winning on stronger evidence so much as triumphing due to better targeting. In some conditions, the personalized GPT-4 had around 80% higher odds of inducing a change of opinion compared to a non-personalized human debater with access to the same information.

Another 2025 paper, by Jasper Timm, Chetan Talele, and Jacob Haimes, asked, if you train an AI model to be more persuasive, what happens to its relationship with the truth? The title of the paper, Tailored Truths: Optimizing LLM Persuasion with Personalization and Fabricated Statistics, does not leave much doubt about the answer. The paper demonstrated that LLMs can use persuasive optimization to generate convincing but false statistics or other misleading framing‚ despite already having the knowledge required for a more truthful answer․

Pattie Maes and her team at MIT have studied the user side of these interactions‚ publishing a paper at the 2026 CHI conference about a month-long study looking at how well people chatting with a chatbot can identify misinformation․ With the chatbot's assistance‚ accuracy increased by 21%; without it‚ the accuracy on new material decreased by over 15% relative to the baseline before the experiment․ The researchers dub this the dependency paradox: the tool improves human judgment when used and weakens it when removed.