EU Parliament votes on AI Act; member states will have to plug surveillance loopholes
Today, the European rulebook on AI - the so-called EU Artificial Intelligence Act (AI Act) - will be adopted in the European Parliament. It aims to regulate AI development and usage in the European Union. Although the Act takes basic steps in safeguarding fundamental rights, it includes glaring loopholes that threaten to undermine its very purpose.
The Act prohibits 'unacceptable' AI uses and outlines technical, oversight, and accountability requirements for high-risk AI in the EU market. It introduces basic safeguards, like mandatory rights assessments, an EU-wide database for high-risk AI applications, explanation rights for those impacted, and transparency measures for powerful AI systems.
At the same time, the Act fails to effectively ban all AI-powered surveillance practices such as automated facial recognition. Specifically, the restrictions on the use of real-time and retrospective facial recognition in the AI Act are minimal and do not apply to private companies or administrative authorities. Since an earlier restriction - that such technology can only be used to address serious cross-border offenses - has been removed from the final text, a vague reference to the "threat" of a criminal offense can now be sufficient to justify the use of retrospective facial recognition in public spaces.
However, the Act leaves the option for member states to adopt more stringent rules on the national level. Together with 16 other organizations, today AlgorithmWatch in an Open Letter calls on the German government to use this leeway and ban biometric mass surveillance without any exceptions.
"In its coalition agreement, the German government has clearly stated that it wants to exclude biometric recognition in public spaces for surveillance purposes. We expect that the coalition parties make use of the discretion the AI Act allows to adopt a full national ban, leaving no room for mass surveillance in public spaces and sending a clear signal to other EU member states."
Pia Sombetzki, Policy & Advocacy Manager at AlgorithmWatch
But even if member states restrict AI-based biometric surveillance, backdoors in the areas of national security, law enforcement and migration remain. A general exemption on national security will allow member states to exempt themselves from the rules for any activity they deem relevant for “national security”. Furthermore, many of the rules set out in the regulation for high-risk AI systems will not necessarily apply to the use of AI by police and migration authorities. For instance, law enforcement and migration bodies will be exempted from public-facing transparency obligations for high-risk applications, which will prevent the public and watchdog organizations from knowing what kind of systems are used by these authorities - despite the high risk of misuse and abuse in this field.
“In areas like law enforcement and migration, where power imbalances are high and the decisions can be a matter of life or death, public scrutiny is an essential mechanism of democratic control. We deeply regret that the EU chose secrecy over transparency."
Nikolett Azodi, Policy & Advocacy Manager at AlgorithmWatch
