Summary: Could AI Chatbots influence a Government’s Decisions?
What does it mean for democracy if our political leaders and government officials allow AI to shape their decisions?
This is a summary of a longer text. Find the full piece here.
AlgorithmWatch has been investigating how the ever-greater use of AI tools, such as chatbots, could have impacts for democracies. One important, but under-considered, route for impact by AI is when it is used by people with power within democracies - our governments, officials, and politicians. This risks the biases and preferences of AI models finding their way, via people with power, into legislation, policy, and other decisions which affect countries and citizens.
In our initial research, presented in full below, we focus on the following questions:
- How do government officials in Germany, Switzerland, and the UK use chatbots to develop ideas, understand topics, and inform decisions about policies and legislation?
- What risks arise from how the chatbots could influence these decisions?
- What safeguards exist, and what are needed?
We identify multiple issues which need addressing. One is that apparent routes to transparency only offer partial stories. Politicians, including the German Chancellor, extol the value of AI chatbots to their work, but scrutiny of how they see the attendant risks is challenging. Parliamentary Inquiries in Germany have not provided further details, but rather references to existing documents and an as-yet-incomplete Centre for AI Competence.
While journalists in the UK had initial success with Freedom of Information (FOI) requests, subsequent attempts have been blocked. In Germany we received a reply to an FOI stating that the German Digital Minister Karsten Wildberger does not use chatbots “in his capacity as Digital Minister” - even though Wildberger himself has spoken of using chatbots for one to two hours per day to structure his thinking. A response to us from the Research Ministry claims that Minister Dorothee Bär also does not use chatbots in her official capacity. We are still waiting, since January this year, on a reply from the Chancellory. In Switzerland, one of the main federal chatbot projects has been reported on in the press but not officially by the government.
We also find, through reviewing literature and testing answers from LLMs to policy questions, that there are potential risks of decision-making being influenced by chatbots. For instance, officials are sometimes advised to add context to prompts, such as “I am briefing Minister XYZ on topic…”.But in LLM tests we find that such contextual information can substantially change the answers chatbots give. In some cases changing details in prompts such as, for example, the name of a Minister reversedthe policy position chatbots presented as “based on the best available evidence”. Without safeguards against such issues, officials' choices of how to write prompts could substantially influence analysis and proposal of policies. This is particularly the case given psychological factors such as “automation bias” (i.e. tendencies to treat machine answers as objective).
Officials need techniques to avoid such risks. These techniques will need to be more complex than simple fact-checking, and involve careful self-reflection and collaborative discussion - which may be challenging if chatbots are being used to provide faster analysis under pressure. Although governments have published documents about safeguards they put in place when using AI - some of which we discuss in the full piece below - they often state requirements such as having “human oversight” without specifying how this would address the issues we raise. We are developing Guidance on use of chatbots which accounts for the risks in more detail, and welcome opportunities to discuss this with people within governments and political parties, or organizations who support such decision-makers.
However such efforts to understand - and support protections against - the influence of AI on democracy are limited without fuller transparency. It is important that there can be independent scrutiny of processes and the methods for limiting risk, not simply high-level promises of accountability and oversight. This topic is complex, wide in scope, and fast-changing. We are limited in our visibility of the full realities of chatbot uses in governments and politics, based on publicly available material. We welcome engagement from others - including officials and politicians themselves - to develop our understanding of risks and possible solutions further.