Introduction

In April 2025, the Trump administration presented a series of tariffs, designed to punish countries that the administration felt had unfair trade imbalances with the US. Many of the numbers involved were bizarre and often nonsensical, including high tariffs on some extremely poor countries that – far from engaging in lopsided trade - rarely buy US products. Various observers reverse-engineered a formula that would produce these numbers, and it quickly became apparent that this formula was the answer that multiple chatbots give to the question of how to 'solve' trade deficits. Despite warnings from the chatbots that the formula was extremely simplistic and came with numerous risks, it seems that the answers were taken and applied by the Trump administration as their official tariffs.

This is an extreme, and admittedly unprovable, example of chatbots influencing governmental decisions. But it reminds us that these tools offer convenient - maybe too convenient - answers to the complex topics addressed by governments and politicians. Some uses are quite high-profile and visible, such as Diella the “AI Minister”, designed to handle public procurement in the Albanian government (and whose likeness was, according to a current lawsuit, taken without consent from an Albanian actress).

Other uses may also be more invisible, but still pervasive. Consider what happens when a government official asks a chatbot to summarize a complex policy area, or a political advisor uses it to brainstorm new proposals. Chatbots make decisions: what to show, what to leave out, what to present as “high quality” or “consensus” and what to critique or dismiss. These are affected by factors like how it was trained and fine-tuned, how the questions were phrased, and other (often unclear) aspects. When used as part of complex systems like a government, these problems can compound.

Such issues may not be well understood by the users. This is true of all users - but what are the implications when such users are people with the power to write laws, or influence policy? We call this group “democratic decision-makers”. In this work we are considering how to understand and address the risks when decision-makers use chatbots to inform their decisions. Politics and government are high-pressure environments with limited resources, multiple complex topics, and often tight deadlines. There may be strong incentives to rely on the speed and convenience of chatbots, even though the risk of subtle influence requires care in safeguards.

Even if “official” guidelines are in place, the attention paid to properly considering the influence of chatbots may be insufficient. Even what counts as “official use” can be unclear. In an example we will discuss further later on, the German Digital Minister Karsten Wildberger has spoken of using chatbots for one to two hours per day to structure his thinking. However, according to a reply from his Ministry to a freedom of information request we sent, he does not use chatbots “in his capacity as Digital Minister” at all. Evidence that chatbots are used by ministers to structure their ideas and systemize information can also be seen in, for example, a study in Lithuania conducted by public service broadcaster LRT.

In this work we are considering what under-addressed risks may exist when chatbots structure the ideas of people in government, and how we can see and mitigate them. We do so with reference to existing literature, attempts to seek data via transparency measures, and some small-scale experiments with LLMs. Although we are interested in democratic decision-makers broadly – including decision-makers in political parties, parliaments, etc. – we have focused at this stage on officials within governments. This is because access to relevant information from governments can be facilitated by material that government officials publish themselves and is available through various transparency and freedom of information rules, often more so than in cases of politicians and advisors who are not in government.

We have focused mainly on governments in Germany and Switzerland, the countries in which AlgorithmWatch and AlgorithmWatch CH are based. We have also added the UK, as the project lead (Oliver Marsh) has past experience working in this government. It is also relatively transparent about its use of technology and AI use in government is being strongly championed by the current administration.

This piece should be read as our preliminary impressions and views based on a combination of sources, not a systematic review. As noted in various sections, there are limitations based on access to information and the scale and fast-changing nature of the topic. Ultimately, the aim of this project is to inspire better practices against plausible risks, potentially via better guidelines or collaboration with relevant partners. This document should be read as an interim update on our research and thinking so far. We welcome input, comments, and approaches from those with expertise and experience to expand on, challenge, and ultimately implement these ideas.

How are chatbots used in Governments

The three governments we studied make use of various transparency registers such as the “Marketplace of AI opportunities” (Marktplatz der KI‑Möglichkeiten) in Germany, the Project Database of the Swiss CNAI (Competence Network for Artificial Intelligence), or the UK Algorithmic Transparency Records. These reveal specific tools developed or procured by governments. Governments and associated bodies also publish various overarching strategies and reports on uses of AI (see this report from the Swiss Federal Audit Office, an independent federal oversight and audit body, or the UK Government's AI Incubator). The fact that we were able to get this information straightforwardly, much of which forms useful evidence for this piece, underlines the value of transparency in the use of AI systems by public administrations - as AlgorithmWatch has long advocated for. However, as discussed further below, the information in these documents only provides a limited picture of how such tools may influence decision-making within governments.

Actors outside government can also use techniques such as freedom of information (FOI) requests and parliamentary inquiries to try and make governments reveal additional information. These can be useful tools for revealing additional or specific information, but can also point out challenges as to how much governments divulge in practice. We discuss FOIs in more detail later. As an example of a parliamentary enquiry, members of the German Die Linke party have repeatedly used the “Kleine Anfrage” system to ask about implementation of AI in the German Federal Government. In its reply to the latest enquiry (from October 2025), the Federal Government largely referred to material it had already published, including the aforementioned Marktplatz der KI‑Möglichkeiten and the German Federal Government Guidelines on use of AI (discussed later in this piece). They also refer to the “Advisory centre for AI” (Beratungszentrum für KI, or BeKI), which is currently still being set up and about which little public information is available.

Finally, public announcements and media reports sometimes reveal use cases. For instance, in a widely-reported speech by German Federal Chancellor Friedrich Merz, he describes trying out a tool – “very specifically in the context of a legislative project that we in the federal government decided on – namely the active pension… It was astonishing what the AI ​​offered, even down to specific wording" – but with no details on the type of tool. Switzerland’s GovGPT, meanwhile, has been mentioned repeatedly in Swiss media, but not on official government websites.

From these sources we can observe a large range of use cases for AI within governments, including statistical analysis and modelling, triaging and sorting internal information, and various other tasks ranging from detection of plastic in water to conversion of old maps and plans into digital form. When we look specifically at chatbots, many of them are designed to interact with members of the public to answer questions or help with access to particular services. However, some chatbots provide research, summarization, and idea generation services for officials. These are tasks in which the chatbot’s role is directly influential on how an official may understand, think about, and make decisions regarding a topic. As such, these are the use cases we focus on here.

Sometimes governments use general-purpose chatbots for such use cases – for instance OpenAI’s ChatGPT, Microsoft’s Copilot, and Google’s Gemini. Governments also develop their own in-house tools. Examples include Redbox in the UK, GovGPT in Switzerland, F13 in Baden-Wuerttemberg or LLMoin in Hamburg, Germany. All offer functions similar to general chatbots, including research and document summarization, but with features such as improved data security or functions like “present the information as a Cabinet Note”. These tools may build on Large Language Models (LLMs) offered by external providers. The Swiss GovGPT is built on the Open-Source LLaMA model from Meta, while the UK’s Redbox was partly developed by Accenture as a “wrapper” around LLMs from OpenAI and Claude.

Chatbot use in governments is a constantly shifting topic. For example, the UK’s Redbox was championed in summer 2025 as a key example of AI in the UK Government, with over 6,000 users and 30,000 messages per week. By October, it was being wound down. We can also see evolving questions around whether to develop in-house (or at least closer-to-home) technologies versus using the tools of, and building partnerships with, large US companies. The idea of “digital sovereignty” in the sense of avoiding reliance on US companies is a component of the German Federal Government Guidelines on use of AI. In contrast, Redbox’s demise in the UK was hastened by “the rise of enterprise solutions like Microsoft Copilot, especially once Copilot Chat became freely available to many government departments”. The UK Cabinet Office is replacing its services with Google’s Gemini LLM, while the Ministry of Justice struck a deal with OpenAI to equip 2,500 civil servants with ChatGPT Enterprise. All these shifting factors mean that any answer to the question “how exactly are governments using chatbots” will change over time. However, the issues we raise in the next section do not require a precise and stable answer to this question; our concerns emerge from broader patterns and principles which seem likely to persist, even as precise practices and technologies change.

Finally, an important question for our work is: What sort of prompts do officials put into the chatbots? The evidence is limited. In the UK, a high-profile FOI request by New Scientist led to the release of Technology Secretary Peter Kyle’s ChatGPT prompts. The small list (7 questions) were relatively simple inquiries, including requests for advice on podcasts to appear on, “Why is AI adoption so slow in the UK small and medium business community?” and “What’s the definition of digital inclusion?”. However, further FOIs on similar topics but asking for broader lists of prompts were rejected by the UK government as “vexatious”. We have not (yet) found similar comparable successful requests in Germany or Switzerland, also not via repositories in Frag Den Staat and Oeffentlichkeitsgesetz.ch.

Inspired by the New Scientist's approach, we have sent our own FOI requests asking for prompts made to chatbots by the Chancellor, the Digital Minister, and the Research Minister of Germany in their official capacities. The Digital Ministry stated that “Minister Wildberger hat in seiner Funktion als Bundesminister für Digitales und Staatsmodernisierung bisher keine KI-Chatbots genutzt” (“Minister Wildberger has not currently used AI chatbots in his function as Federal Minister for Digital and Government Modernisation”). However, Wildberger himself has stated in an interview with Die Zeit that he uses AI chatbots, in particular Claude, “Oftmals ein, zwei Stunden am Tag… um Gedanken zu strukturieren” (often one to two hours per day, to structure thoughts). He relates that he often provides it with unstructured thoughts and asks for structure and two to three additional ideas, then reflects further and prompts again, often four or five times (“'Strukturiere mir das bitte, gib mir noch zwei, drei Ideen.' Dann denke er darüber nach und spreche es noch einmal ein. ‘Meistens sind das vier, fünf Schleifen'”). The Research Ministry supplied a similar answer regarding Minister Dorothee Bär. At the time of writing, we are awaiting an (overdue) answer from the Chancellery and also plan more requests, including in Switzerland.

Even in the absence of clear answers, this example makes a clear point. The thoughts of the Digital Minister are relevant for how he makes decisions and shapes the decisions and activities of his Ministry. Chatbots are likely playing a role in this thinking. It is hard to see how such use is not part of his function as Digital Minister. Most likely, a similar activity is happening among many of his officials as well as politicians and officials across other Ministries and in other countries. There are democratic questions to be asked around how these tools are “structuring the thoughts” of those in positions of power.

The risks

To re-state our main interest: We are considering potential impacts when chatbots are used by democratic decision-makers in ways which may shape their understanding and/or their decisions made with regard to an issue.

To be clear, first, about what we are not claiming: We are not claiming that democratic decision-makers are totally outsourcing their thinking or decision-making to chatbots. Nonetheless we should still ask about the influence of chatbots on decisions, especially if that influence may not be fully clear even to the user. The evidence we cite in the rest of this piece raises various issues around potentially complex biases, how minor changes in prompts can result in large changes in outputs, how sources are selected and presented (or not), etc. Even if a user is “fact checking” an output to avoid incorrect information or clear biases, they may not be fully aware of all these other (potentially quite subtle) issues.

Nor are we focused on whether LLMs and chatbots bring new issues into governments. Intransparency, bias, and unclear influences on decision-making have always been risks in government decision-making. The question is whether new technologies reproduce or address these risks. AlgorithmWatch’s mission is to “fight for a world where algorithms and Artificial Intelligence (AI) do not weaken justice, human rights, democracy, and sustainability, but strengthen them”. New technology should not reproduce risks of the past, but rather help to address them.

Research

The views we present below are informed by various strands of research. Firstly, we reviewed guidelines and proposals on safer AI use in government, with a focus on the UK, Germany, and Switzerland, plus the EU level. This included “official” guidance published by administrations themselves and also “unofficial” guidelines such as the Bennett Institute's guide "Using LLMs responsibly in the civil service" and training documents by the government support organization Apolitical. We reviewed academic and scholarly literature on implications of LLMs for government and democracy. We also looked at papers reporting experimental research into biases in LLMs. While often not specifically dealing with our question of chatbot use for decision-making in government, this collected academic literature gave us a series of useful concepts to help frame our thinking and identify gaps in guidelines; we discuss some examples below.

To note: As with the general topic of AI use in government, the range of such guidelines is growing at a rapid pace – see for instance Apolitical’s 2025 report on the range of material created in just two years for its “AI Campus”. As such, again, we do not claim to be systematically summarizing all relevant material.

Finally, we also conducted small-scale tests of LLMs to see how they responded to the sort of prompts that officials might write. This was an initial and small-scale research, not a conclusive quantification of how often, or under what conditions, certain patterns occur. However, the method used could be applied to a wider range of prompts in the future, in particular if we can access examples of “real world” prompts made by government officials via FOI requests or other transparency methods.

From Obvious Flaws, to Subtle Bias, to Wider Risks

There are some obvious, surface-level risks to using chatbot answers to inform decision-making. Two in particular are commonly mentioned in government guidance. The first is that they can produce factually inaccurate answers, often called “hallucinations”. The second is that they can be “biased”. Bias is a complex term and its meaning was not always spelled out in documents we reviewed; when it was, it generally referred to discrimination against particular groups or views. Such discrimination is important and plenty of work, including from AlgorithmWatch, has illustrated its social impacts in numerous contexts (we continue to record cases via our reporting form). We consider it a positive that government guidance is warning officials against these as standard practice.

However our concerns relate to deeper and more subtle issues, which may not be spotted simply through checking for false information or potential discrimination. Particular types of biases that can arise in chatbot answers include, for example:

• Sycophancy, where models adjust answers to adapt to the preferences (explicit or implicit) of the user.
• Source-attribution effects, where identical content is treated differently when attributed to different sources.
• Tendencies to present institutional assumptions as neutral facts.

These biases can have downstream consequences when humans and chatbots interact to develop ideas and proposals. Academics have considered various processes by which the regular use of conversational AI tools gradually becomes incorporated into an individual's analytical thinking patterns, potentially influencing what they consider as “good” or “relevant” information (“cognitive integration”, Chiriatti et. al., 2024) and downstream beliefs (“belief offloading”, Guingrich et al. 2026). Similar ideas have also been expressed in the idea of “automation bias” - that people can too easily accept answers from automated systems. A paper on Better Policymaking in the Age of AI (Medaglia 2025) describes how:

“In policy evaluation, automation bias occurs when evaluators accept AI-generated findings without sufficient critical scrutiny, sidelining contradictory evidence from qualitative insights or local expertise (errors of commission); or when they fail to notice problems or seek alternatives because they assume the automated system must be right (errors of omission)”.

These biases can compound, particularly when chatbots might be used across multiple decision-makers within an organization. Institutional assumptions may become embedded in how officials phrase questions and critically evaluate answers. What looks like consensus may actually reflect convergence across answers generated by chatbots, if all the officials are prompting the chatbots in similar ways (Guingrich et. al. 2026). If chatbot answers are being handed up chains of authority, to senior officials and then to ministers, a risk is created that no one considers trade-offs or interrogates why some principles have been elevated over others (Institute for Government, 2025). Such macro-level problems can emerge in the absence of “formal institutional frameworks, oversight or guidance”, a phenomenon referred to as the “shadow use” of generative AI in public administration (Tangi et. al. 2025). But these risks are also connected to a micro-level question of how officials prompt chatbots, which we consider in more detail in the next section.

When Officials Prompt Chatbots

Some of the guidance and training programs we analyzed advise certain prompting behaviors, sometimes even offering model prompts. Many of these advise adding contextual information to prompts. This can relate to the intended form of output (“be concise”, “cite specific evidence”) and/or the conditions in which the output is being produced (“this is for Minister X”, “act as an experienced policy researcher in social welfare”). However, adding further context to prompts can actually heighten risks, sometimes in surprising ways. Some research has suggested, for example, that asking chatbots to “be more concise” can lead to more inaccuracies. 

We therefore conducted small-scale tests of LLMs, seeing how they responded to the sort of prompts that we expect officials might write, such as formulating positions or summarizing evidence. We say “the sort of prompts we expect officials might write” because, as noted in our discussion of FOIs, getting “real world” examples of prompts is challenging. Some guidelines do provide suggestions (even templates) of how officials should write prompts, often suggesting additional contextual information (such as “I need to brief Minister XZY”). However, we cannot be sure how often this advice is actually used in practice. Nonetheless, the prompts we developed generally did include additional contextual information of the prompts suggested in guidance.

To test the effects of this type of contextual information, we varied it – for instance using names of different ministers, varying sources of arguments, or phrasing requests differently – and observed how answers changed. In order to test a wide range of possibilities and analyze numerous variants, we queried the LLMs multiple times via automation. This was done using the official application programming interfaces (APIs) provided by the companies and assessing answers through a combination of LLMs and manual human inspection of some answers (enabled by customizing the PETRI framework). For more information see Loi 2026a.

The aim here was not to systematically investigate or map particular changes; there were too many variations for that to be straightforward. This was rather done to model the effects of adding contextual information to better understand any related risks - focusing on the kinds of contextual information that officials might be expected to use. We found out that sometimes, variations in prompt language did change outputs in ways that (i) impact important and substantial parts of answers, such as assessments of evidence, and (ii) may be hard to predict and mitigate a priori.

As an example of (i), two specific examples are provided below. These illustrate how providing contextual information about different audiences and their expectations can reverse an assessment of empirical evidence – even when a potential safeguard is included (repeatedly asking for an “evidence-based” answer and wanting to “acknowledge uncertainties”).

In the first example, the contextual information is that the author is addressing a particular prominent politician, from either the SPD (centre-left German party) or the CDU (centre-right German party). In this case, switching which politician is addressed reverses the answer to the question of which policy position “evidence most strongly supports”.

To be clear, this was not a universal pattern; there were also cases where similar changes of information did not have a similar effect, or where the chatbot explicitly recognized it was being asked to support particular presumptions.

LLM Tests Examples

The second example illustrates point (ii) above that the variations were sometimes surprising and hard to predict a priori. We asked LLMs to assess the credibility of a certain policy-related argument, and investigated how the assessment of credibility changed when we switched only the cited source of the argument.

As well as the LLMs substantially changing the evaluation of an argument simply because the source changed, these examples illustrate a phenomenon of “coherence bias”, in which the LLMs sometimes downrated credibility of an argument if it came from a source that would normally make the opposite argument. This can also be seen in outputs from other experiments we ran, for example:

“This argument presents a coherent case for fiscal conservatism with legitimate empirical foundations… However, the position appears somewhat inconsistent with the SPD’s traditional stance on public investment and social spending… which weakens its overall credibility given the source.”

or

“the position appears notably inconsistent with the Friedrich-Ebert-Stiftung’s typical advocacy for increased public investment and social spending to address inequality and infrastructure deficits… the source attribution raises questions about the authenticity or representativeness of this particular stance.”

This phenomenon of coherence bias may be surprising and therefore hard to predict a priori. It runs contrary to an expectation that many humans may apply in analyzing arguments: That an argument may actually be more credible when it comes from a source that would normally adopt the opposite position. I.e., this argument must be so strong that it has even convinced its critics. The importance of this in technical LLM analysis is discussed further in Loi 2026a.

For the purposes of this current work, our point is that variations in important claims – such as what positions are better supported by evidence or whether certain arguments or sources are more credible – can vary in chatbot answers based on how a question is phrased in ways that may not be expected or predictable by users.

Safeguards

Chatbots should not be used as straightforward “question in – answer out” machines. Nor should checks on them be limited to simply checking for accuracy or obvious biases, but also critically reflecting on how the chatbot’s answer is a limited slice of reality, conditioned by word choices in prompts. It is important in the context of democratic decision-makers to ensure this feature does not narrow perspectives.

Guidance should not present bias just as a product of an LLM’s training data or instructions on how to behave; bias also emerges from how officials use these tools. Officials should challenge any of their own assumptions, which may be reflected in how they frame questions or other prompts. But as it may be hard to a priori predict the ways in which a given prompt might affect an answer, we support proposals – which for example appear in the Bennett Institute’s guide to LLM Implementation or the German Guidelines for the Use of Artificial Intelligence in Federal Administration – that officials should explore multiple prompts for any given task or question.

Use of chatbots within teams and wider organizations can, in principle, support the proposals mentioned above. By bringing a wider range of people and perspectives to the task, implicit assumptions can be challenged and a wider range of ideas (for prompts, for challenging answers, for testing alternatives) can be developed. However, this requires proper transparency and structured collaborations. These benefits will only accrue if chatbot use is properly recorded, open for inspection, and properly discussed. Such behavior should be common practice in any workplace where AI is used, including in governments. However, some evidence from literature suggests that “autonomous and often undocumented use… is already widespread and raises important questions for public administration” (Tangi et. al. 2025).

Much of the guidance we reviewed considers the question of how chatbot use should be “embedded” in a wider organization in ways which ensure accountability. A common theme is visibility of how AI is being used, sometimes with labelling proposed, and references to a “human in the loop” approach. However, this can sometimes be limited to simple awareness of AI use, not critical reflection. For example, the UK AI Playbook requires “meaningful human control at the right stages” and instructs officials to "review and validate AI outputs" — but assumes the reviewer can identify what the AI contributed. The German Federal Guidelines call for "traceable results" and "human oversight" while leaving authorities to decide which steps can use AI "without jeopardising traceability."

Similar issues – calling for human oversight without specifying what that entails – appear across other documents we also reviewed, including from the European Commission, Court of Justice of the EU, and the European Data Protection Supervisor. Some promising steps come from the Swiss National Centre for AI Competence (CNAI), which has stated that officials "must be able to justify at all times any decisions made on the basis of results from generative AI tools" – though we would still like to see more concrete proposals as to how. Across the sources, "oversight" is too often treated as a principle to affirm rather than a practice to specify: officials are told that they should review, not how to detect biases that pass surface-level fact-checking.

To many of these oversight mechanisms we can pose the following problems/questions:

1. Accountability for what? Accuracy alone misses the point. The biases we identified produce accurate-but-skewed outputs. The question is: whose choices does this reflect?
2. What should visibility reveal? Prompts? Outputs? Changes made? Reasoning? Without specifics, "visibility" becomes a checkbox exercise of whether AI was used or not.
3. What does "human in the loop" actually require? Skimming and approving technically satisfies "in the loop" — but that's rubber-stamping, not oversight. More specificity is needed to address the risks of influence we have laid out in this piece, in addition to other ethical questions.
4. Labelling for whom, and to what end? What should readers do with the label? If "be aware" is the answer, that's not a safeguard.
5. How do we detect collective risks? Risks such as chatbots converging on similar (but skewed) answers, are invisible to individual review. Safeguards must compare AI use across users.
6. Does documentation change behavior? Or just create paperwork? Effectiveness depends on whether records are used, not just stored.

There should be guidance supporting officials to record, discuss, and consider how chatbots may be influencing outputs. As noted above, some existing guidance makes moves in these directions, calling for officials to test multiple approaches (rather than accept the chatbot’s first answer) and justify their thinking. But there is currently not enough specificity and transparency to give confidence that the influence of chatbots is being sufficiently understood and that risks are mitigated.

Conclusions and Future Work

Various studies and documentation, from researchers and from governments, suggest that there is awareness that government use of AI creates risks, and there are also various proposals for safeguards. However, taken together, our current evidence does not show substantial attention paid to the problem of how to understand and mitigate the subtler risks when chatbots are used to help “structure the thoughts” of democratic decision-makers. Moreover, what proposals exist seem to rely too heavily on oversight practices which are not specified or transparent.

Regarding next steps, our first interest is to understand how to increase knowledge around actual, real uses of chatbots in policymaking – not broad cases in government documentation or occasional media reportage of Ministerial uses. We aim to test transparency provisions such as freedom of information requirements and disclosures proposed under AI regulations and guidelines. But we are also interested in working with partners who can provide further insight into the relevant working contexts – including politicians, political advisors, and officials themselves. This way, we can better understand these uses in context, and the needs of officials to provide themselves with appropriate safeguards.

Such understanding can support the next planned step to this work – considering how to take on our own challenge and present guidance which would offer the sort of specificity and accountability we call for above. Tentatively, we are working on three principles and consider how these can be operationalized in practice:

• First principle: Treat every AI output as a first draft that reflects choices you didn’t make.
• Second principle: You must show how you challenged or accepted the AI’s choices.
• Third Principle: The organization must treat AI upskilling as a collaborative exercise of recording and discussing experiences.

Again, opportunities to discuss these with people and organizations closer to “the coalface” of democratic decision-making would be welcome to orient such guidance towards the practical realities of these settings. There is room to learn from approaches recording and scrutinizing uses of other new technology in decision-making, such as WhatsApp (as discussed by the Institute for Government).

In conclusion, however, we must note: This work is taking place in an environment of increasing hostility towards AI accountability, including amongst some democratic decision-makers themselves. For instance, we would argue that the sort of AI upskilling proposed in our third principle would be a positive example of proposals under Article 4 of the AI Act: That providers and deployers of AI systems should take measures to ensure a sufficient level of AI literacy of their staff. However the EU’s proposed Digital Omnibus may take this responsibility off organizations, instead requiring Member States and the Commission to generally promote AI literacy.

In the absence of legal requirements for safeguards, it may be that our proposals are limited to voluntary safeguards for particularly diligent democratic decision-makers. Others who are less keen may reply: Such safeguards add work and create friction, and democratic decision-making is already highly challenging and often bureaucratic. Tools based on LLMs – whether chatbots, AI summaries, or other products – are designed to straightforwardly turn a need into an output, at greatly increased speed and with greatly reduced effort. As noted in our introduction, such offers are likely to be particularly appealing to democratic decision-makers facing fast-moving and complex work. Indeed, evidence suggests “efficiency” is the main benefit many officials see in generative AI used in public administrations (Tangi et. al. 2025).

But safeguards, and the frictions that comes with them, are integral parts of democracy – and attempts to reduce friction can contribute to many of the issues modern governments face. Proper processes for safeguarding and the transparency that ensures they can be externally scrutinized are not a nice-to-have; they are essential parts of the relationship in democracies between citizens and the state. Assessing risks when chatbots might influence decisions by politicians and officials – risks which are complex and context-specific – from the outside with limited information is challenging. We are continuing this work, as well as developing our own proposals, but more information is needed. We welcome engagement from others – including democratic decision-makers themselves – to understand risks, build safeguards, and ensure use of AI supports rather than undermines democracy.