Artificial intelligence models can now generate answers to political debates that everyday people find more authentic and relevant than the actual answers given by real-world politicians. This provides evidence that modern technology can easily be used to imitate public figures and spread misinformation that voters easily believe. These findings were published in the journal pro swan.
Recent advances in artificial intelligence have led to systems that can generate highly realistic text. Programs such as GPT, Claude, and Gemini fall into the broad category of generative AI. This means that it is a software system designed to create new content based on the vast patterns it learns during training. These models can teach students to write persuasive essays, summarize legal documents, and even imitate certain writing styles of human authors.
Because these systems are very good at mimicking human communication, they pose a potential risk to the public information realm. Bad actors could use this technology to create fake statements that look exactly like real politicians’ statements. This risks contaminating political discussions with fabricated content. Such deception can confuse voters and threaten the cohesion of society as a whole.
The authors of the new paper wanted to test whether ordinary citizens can tell the difference between real political speech and machine-generated political speech. They also wanted to measure how the public perceived the quality of computer-generated responses compared to human voices. Understanding how people react to spoofed political content will help scientists better understand the risks posed by these technologies.
To investigate this topic, researchers focused on Question Time, a popular political debate TV show broadcast by the BBC in the UK. The show features a panel of politicians, businesspeople and journalists answering topical questions from viewers. The authors collected 520 questions from real audiences and responses from 112 different celebrities. These episodes originally aired between June 2020 and November 2021.
The researchers then used an artificial intelligence model called GPT-4 Turbo to create fake responses to the exact same audience questions. They instructed the software to role-play as a particular celebrity who answered the question first. To enable the machine to imitate a person, the software received a short biography taken from the first paragraph of a famous person’s Wikipedia page. The machine was instructed to answer questions directly, use a conversational tone, and keep responses to about 200 words or less.
With both real and fake responses, the researchers gathered a representative sample of 948 British adults to rate the texts. Study participants were told to read a transcript of a television program discussion. At this stage, they were not told that half of the text they were about to read was computer-generated.
Researchers divided participants into three separate test groups. In the first group, participants read the audience questions, saw the speaker’s name, and read one response. This response can be either a real response or a machine-generated response. Next, participants rated the trustworthiness of the texts. This means you have determined how likely it is that the speaker actually said it.
In addition to reliability, this first group evaluated responses on two other metrics. They assessed the coherence of the text, i.e. the logical flow of the argument and internal reasoning. We also assessed the relevance of the texts. This measures how well the statement actually answers the original audience question.
In the second group, participants were shown both the real and spoofed responses side by side. They had to compare two texts on exactly the same scale: authenticity, coherence, and relevance. We also needed to determine whether both statements had the same basic meaning or expressed completely different points of view.
The third group saw audience questions, names of famous people, short biographies of people, and real or fake answers. This helped researchers determine whether knowing the speaker’s background changes the way people judge the trustworthiness of a text. As a control test, some participants in this group observed actual responses paired with completely random and mismatched speakers.
Results from all three groups consistently showed that participants preferred machine-generated text. Participants generally rated the fake answers as more authentic, more consistent, and more relevant than what real celebrities said. Even when the fake text was placed right next to the real text, participants tended to believe that the artificial response was a more authentic statement.
The researchers noted that machine-generated answers are especially good at staying on topic. Real politicians often duck difficult questions during live debates, so their answers don’t seem to mean much to viewers. The artificial intelligence acted on instructions to answer the prompts and directly addressed the questions. Furthermore, the machine-generated text had a much higher percentage of words that overlapped with the audience’s questions.
When looking at the content of what was said, participants felt that the real and fake texts conveyed different messages about half of the time. The authors analyzed a subset of these discrepant responses in detail to understand the differences. They found that in about 26 percent of these cases, the computer generated positions that were completely different from the real politicians’ actual views. This provides evidence that this software can create believable statements that completely misrepresent a public figure’s political platform.
The researchers also analyzed the linguistic style of the text to see how computer writing differs from human speech. Human speakers were found to use more epistemological markers such as “I think” and “In my opinion.” These markers indicate that a person holds a particular subjective position on an issue. Human speakers also used discourse markers like “because” and “in the first place” more to connect their ideas during live speeches.
Artificial intelligence tended to use a broader vocabulary than human speakers. Also, nominalization, the grammatical process of turning verbs and adjectives into nouns, was further used to make sentences sound more abstract and formal. Despite this visible difference in language style, human readers did not seem to notice it. The difference in writing style did not reduce the credibility ratings of the fake texts at all.
At the end of the experiment, the researchers surveyed participants about their thoughts on technology in politics. Initially, they asked these questions without disclosing that AI was used in the research. Most participants said they were familiar with AI and generally supported its use in public debate, as long as the process was transparent.
The researchers then revealed that the participants were just reading a computer-generated text and were asked the same survey questions again. More than 90% of participants did not change their opinion even after learning the truth. But those who changed their minds tended to realize they didn’t know more about AI than they originally thought. Many participants expressed surprise and amazement at the quality of the fake responses using the optional comment box.
Although this study provides robust data on how people perceive machine-generated text, it also has some limitations. This study focuses on only one debate television program from a particular country. It also relied on a single artificial intelligence model. British viewers’ reactions to transcripts of particular BBC programs may not be representative of how people in other countries engage with different types of political media.
Another potential limitation relates to the nature of live debate. Human speakers were answering questions on the fly, under pressure and sometimes interacting with a hostile environment. Naturally, this will result in less sophisticated writing. People may have found the synthetic text more authentic simply because it read more like a heavily edited press release than a live, unscripted statement.
Future research should investigate how these generated statements perform in other contexts such as social media posts and prepared speeches. The authors suggest that there is an urgent need to study targeted misinformation. Scientists need to know whether the public would still believe the statements produced if the machines were specifically instructed to produce extreme or polarizing political views. Understanding these dynamics will help society prepare for the influx of synthetic media in future elections.
The study, “LLM-impersonated discussion posts are more authentic, relevant and consistent than the originals: A representative study using BBC1’s Question Time,” was authored by Steffen Herbold, Alexander Trautsch, Zlata Kikteva and Annette Hautli-Janisz.
