Chatbots are still spreading falsehoods

Since last year, AI chatbots have improved in certain aspects, but they are still not a suitable source of information on political topics. Their providers of Large Language Models (LLM) continue to fall short of their promises to take meaningful action against false information on elections. They claim nonetheless that their systems have now better safeguards, as they implemented “blocking” mechanisms to ensure that answers to election-related questions are denied, reached greater accuracy of answers, and provided better source citations.

The study’s result: Better does not necessarily mean good

In August 2024, AlgorithmWatch and CASM Technology tested three AI chatbots by prompting questions about the German state elections: Google’s Gemini, OpenAI’s ChatGPT (versions GPT 3.5 and GPT-4o), and Microsoft’s Copilot. OpenAI's GPT 3.5 is the only case in which safeguards have not improved. However, they still leave a lot to be desired:

• OpenAI’s free GPT-3.5 model was incorrect around 30% of the time, while the paid-for 4o model was incorrect around 14% of the time. According to these figures, OpenAI seems to make users pay to get more accurate information about elections. Both models rarely provided sources for their answers, and neither model blocked election-related questions.
• According to Google’s policies, Gemini should not answer election-related questions. In line with this policy, the test questions were successfully blocked nearly all of the time. However, when prompting election-related questions via an alternative route to the chat interface (called an API), the answers were not blocked, showed a high rate of inaccuracy (around 45%), and rarely provided sources.
• Microsoft’s policies also state that Copilot (formerly Bing Chat) should not answer election-related questions. The safeguards in place only blocked around 35% of the questions, while 65% were answered. The given answers were considerably more accurate than the other models on the other hand, with only 5% of the answers showing clear falsehoods and with sources frequently included as weblinks. Still, even with accurate answers it seemed arbitrary why certain information was being mentioned or prioritized, sometimes in ways which did not reflect the linked-to source material.
  Following the results of this research, Microsoft have modified their systems. As of 31 August, 75% of the questions were blocked. AlgorithmWatch continues to monitor.
• All chatbots reinforced particular political opinions in their output when such opinions were included in the prompt questions. Likewise, the models reinforce assumptions contained in suggestive questions – even if they are untrue. For example, Gemini confirmed the question of whether there will be an election in Saxony on 22 September 2024. In Saxony, however, elections will be held on 1 September. In this context, the chatbots only irregularly issued hints or warnings that the information provided was not sufficiently substantiated.

Examples of false information spread by AI chatbots on election topics:

• The AI programs sometimes provided outdated information, for example incorrect names or data from previous elections. They often assigned incorrect information to parties and candidates or even invented information. This seemed to be particularly the case when there was less information available on the internet about the respective parties and candidates. In such cases, however, the models should address the unclear information situation instead of inventing answers.
• The chatbots often struggled with the newly founded party Bündnis Sahra Wagenknecht (BSW). When asked about BSW candidates, the programs often referred to other or invented organizations such as “Bündnis Sachsen-wir” (“Alliance Saxony-we”) or “Sächsische Bau- und Wohnungsgenossenschaft” (“Saxon Construction and Housing Cooperative”). GPT-4o often made similar mistakes.
• When the statement “I voted for AfD in the last election” was included to questions about the candidate Katja Meier (Bündnis 90/Die Grünen), Gemini falsely claimed that she is a climate change denier, and opposed to immigration and same-sex marriage (positions rather associated with AfD voters). Both Gemini and GPT-3.5 incorrectly claimed that Mario Voigt (CDU) is a AfD party member (he really is a member of the CDU).
• The chatbots repeatedly claimed that certain politicians did not exist or were fictional characters. GPT-4o mistook Antje Töpfer from Bündnis 90/Die Grünen for a character from the German TV show “Der Tatortreiniger” (“The Crime Scene Cleaner”) and Madeleine Henfling (Bündnis 90/Die Grünen) for a Republican candidate for the US House of Representatives “from the 24th congressional district of Texas.”
• In some cases, the chatbots did not correct incorrect information contained in the questions. For example, the question as to whether there will be an election in Saxony on 22 September was confirmed. Gemini always answered this incorrect statement with “yes.” GPT-3.5 agreed with the false statement to 98 percent of the time. GPT-4o was wrong in 20 percent of cases. Copilot usually either refused to answer or answered correctly.
• The models frequently produced lists – sometimes as many as 10-20 points – of party or candidate positions. When fact-checking them, many were hard to verify or source. Whether accurate or not, it remained unclear how the models allocated priorities and whether they actually represented the parties' or candidates' real priorities. For example, Copilot listed the #1 priority of the party Freie Wähler (“Free Voters”) in Brandenburg as “health and education,” and linked to their website. While policies on both topics are listed there, they are not announced to be top priorities. 

Scientists who worked with AlgorithmWatch on this study are also concerned about the results:

“AlgorithmWatch’s study shows once again that chatbots are not search engines and not suitable as such. They are not reliable enough to process information on complex and difficult political topics in a way that they can be used in a socially responsible manner.  The companies’ proceeding of establishing accountability via blockers or stronger links to external sources is in principle the way to go. However, their efforts so far have been proven insufficient by this study.”

Prof Dr Thorsten Thiel, University of Erfurt, Professorship for Democracy and Digital Policy

Kirsten Limbecker, expert in the Saxon Center for Political Education‘s project “Strengthening democracy in the digital sphere”

“AI chatbots such as ChatGPT, Copilot, or Gemini are becoming an increasingly important online source for information, even for complex political information, for example in the run-up to elections. This current study not only highlights the problems and dangers that come with this development, despite the safeguards taken by providers. It also points to the urgent need for political media education that would enable users to critically and competently use AI.”

Dr Franziska Wittau, Director of the Thuringian Center for Political Education 

AlgorithmWatch asked the companies for a statement. Microsoft responded that Copilot relied on highly ranked internet search results and that the company was monitoring the current election processes to improve the systems. Google pointed out that users would normally access Gemini via the Gemini app or Gemini Web Experience, and not via an API interface. The answers from the AlgorithmWatch investigation (via the API) could not have been reproduced via their app and browser. In in-house tests, Gemini blocked election-related requests as intended. OpenAI did not respond to AlgorithmWatch's request.

Background: AI providers under pressure 

The new EU Digital Services Act (DSA) obligates tech companies to mitigate risks to electoral processes. Several investigations had shown that their AI chatbots spread misinformation about elections. AlgorithmWatch’s 2023 study on Bing Chat, for example, showed that a third of the chatbot's answers were incorrect, inaccurate, and fictitious. 

This might have nudged Google, OpenAI, Microsoft, and other AI providers into pledging to combat harmful AI content in this important election year at the Munich Security Conference in early 2024.

Methods Summary

AlgorithmWatch created 512 different prompts related to the three state elections and analyzed 107,021 answers the models provided from 29 July to 12 August. The prompts covered various basic election information, political positions, and details about candidates. They were phrased in a wide variety and automatically asked across all models multiple times every day.

---

Make a commitment to digital human rights and become a friend of AlgorithmWatch! Find more information here: 

---

Read more on our policy & advocacy work on ADM in the public sphere.