The draft report includes some significant steps towards a more effective protection of fundamental rights: More obligations on deployers (aka “users” in the terminology of the Act) of AI systems, such as the duty to inform affected persons of the use of high-risk AI systems. To allow for public scrutiny and accountability, it is key that not only high-risk AI systems but also uses thereof are made transparent. The draft report considers some of our core demands with regard to the planned EU database:
- ✅ When a public authority uses a high-risk AI system, it must register it in the database.
- ✅ The database shall be user-friendly, accessible, easily navigable & machine-readable.
- ❌ High-risk AI use cases by private sector are not included.
- ❌ Not ALL use cases by public authorities must be registered (only the high-risk ones)
The definition of AI remains without any significant changes. A positive development, as the current definition is rather broad and should be kept so that systems which can have a significant impact on our lives are in the scope of the Act. Fortunately, the draft report did not follow Axel Voss’ direction (European Parliament Committee on Legal Affairs) in excluding the area of national security in the scope of the Act. This would leave major loopholes, as many AI systems could be deployed in the name of “national security”.
Further, at least some uses of predictive policing systems shall be banned – unfortunately, not all of them. However, these systems have been demonstrated to infringe fundamental rights, which is why AlgorithmWatch alongside many civil society organisations call for a ban of predictive policing systems in the EU.
We welcome that people affected by an AI system – that falls under the scope of the Act – shall have the right to lodge a complaint against the providers or users of that AI system. This is key!
However, the report does not consider the impact of AI systems on the environment & aspects of ecological sustainability. We recommend that it includes transparency indicators about the resource consumption of AI systems, which is an imperative to combat climate change.
The draft report also misses the opportunity to close loopholes with regard to the ban on biometric recognition in public space – and leaves the proposed ban as limited as before.
As next steps, members of the two main committees (IMCO-LIBE) can table amendments until May 18. Meanwhile, we will work together with civil society partners on the strengthening of the Act to ensure our rights are protected.
A detailed analysis of the draft report can be found here.