The more certain an AI lie detection system is to accuse someone of deception, the more likely people will distrust it, and this distrust will actively undermine the AI’s accuracy. This new research Computers in human behavior.
Artificial intelligence (AI) has proven to have an uncanny ability to spot deception. By analyzing patterns in written language, AI models can classify statements as truthful or deceptive with an accuracy far beyond what humans can typically achieve on their own. In fact, research has shown that trained professionals, such as civilians and police officers, perform only slightly better than chance when trying to detect lies.
Nevertheless, experts argue that such high-stakes decisions should not be left to AI alone, especially in legal and forensic contexts where they can have significant implications for defendants. As a result, there is a focus on “hybrid” systems in which humans review and, if necessary, override AI predictions.
A new study investigated how people react when placed in such a supervisory role. The researchers were particularly interested in whether trust in AI lie detectors is influenced by two important characteristics: the overall accuracy of the system and the confidence expressed in individual judgments.
The team, led by Ricardo Loconte from IMT at the Lucca School of Advanced Studies in Italy, recruited 373 English-speaking participants (52% female, average age 39 years). Participants were randomly assigned to one of two conditions. I was informed that the AI I was working with had a relatively low accuracy of 54% or an accuracy of 89%.
Each participant then read 10 short passages about real-life experiences, such as being hospitalized or being in a car accident, and were shown the AI’s predictions for each one. This was displayed on a sliding scale indicating both the direction of the decision (truth or deception) and the AI’s confidence level. Participants then rendered their own verdicts using the same criteria.
While the results confirmed that people were more likely to follow high-precision AI, they also revealed a striking and counterintuitive pattern. The more confident the AI was in determining a statement as a lie, the more the participants objected to that verdict and changed their judgment toward viewing the statement as true. This effect also extended in the opposite direction. When the AI confidently labeled a statement as true, participants became more suspicious and shifted their judgment toward deception.
As the authors note, participants tended to deviate from AI predictions when they were made with high confidence, “particularly when the model predicted deception,” and when the predictions were from low-accuracy models.
Importantly, this skepticism actually came at a cost to overall accuracy. When used in conjunction with precision AI, participants achieved a detection rate of 76%. This is significantly better than chance, but still well below 90% of AI’s standalone performance. Combined with the lower accuracy model, it performed similarly to the model itself (accuracy 57% vs. 54%). This means that human input also did not provide any significant improvement. In both cases, human involvement provided no added value and performance was significantly degraded in high-precision scenarios.
The researchers suggest several explanations for why people resist confidently accusing AI of lying: “Participants (tend to) overestimate their own deception detection abilities. Additionally, truth-default theory (strong reason to suspect otherwise) Given the social cost of falsely accusing someone of lying, such aversion may also reflect users’ wariness about false accusations, given the social cost of falsely accusing someone of lying.
This study has notable limitations. For example, the task was completely hypothetical, and participants were not provided with any real incentives to make accurate judgments. Additionally, all statements were presented in text only, removing the conversational context that typically informs judgments of deception.
The study, “Humans mistakenly reject confidently condemning AI decisions,” was authored by Riccardo Loconte, Merilyn Monaro, Pietro Pietrini, Bruno Verschule, and Bennett Kleinberg.
