An English police force created its own ethics committee and it’s totally not ethics washing, they say.

In early 2019, the West Midlands Police and Crime Commissioner (PCC), the elected official overseeing the West Midlands police force, created an Ethics Committee. The Committee was to provide advice to the Commissioner on the works of the police force’s “data analytics lab”.

This lab leads several projects involving algorithmic policing. One is about mental health, for instance. The lab built a model that attempts to predict how many mental health emergencies a given ward will have to handle in a given period. (It’s “not obvious how [the model] showed new insights”, a member of the Ethics Committee noted).

But the lab’s flagship project is the National Data Analytics Solution, or NDAS. The solution, which is developed in partnership with several other police forces in England, aims at bringing together vast troves of data from the police, as well as from health and education authorities, related to suspects or criminals. NDAS should let police officers detect people likely to commit crimes, too.

The Ethics Committee, which convenes every quarter, has taken a critical look at NDAS and other algorithmic policing tools put forward by the West Midlands police. In its first meeting, in April 2019, it questioned the reliability of an algorithm intended to detect people at risk of reoffending. In particular, it criticized that the police used stop-and-search data in its model, thus creating a self-reinforcing feedback loop. The police subsequently adapted the model, so that it now only takes into account stop-and-search data when the search was “positive”, but it remains unclear exactly what stop-and-search data are used in the NDAS model.

“Extremely proud”

Such exchanges are proof that the Committee is working as intended, Tom McNeil told the Guardian at the time. Mr McNeil, then strategic adviser to the PCC, was one of the main driving forces behind the creation of the Ethics Committee. He is now assistant Police and Crime Commissioner and told AlgorithmWatch that he was “extremely proud of the work of our ethics committee”.

Indeed, the Committee’s minutes and supporting documents, which are available online, provide comprehensive insights into the projects of the West Midlands police and the reactions of Committee members. It seems that the police and the Committee engage in good faith with one another and that advice and criticism is investigated and taken seriously.

Marion Oswald, a Senior Fellow in Law at University of Northumbria and chair of the Ethics Committee, told AlgorithmWatch that the set-up could be replicated with other local police forces, even at the national level.

Based on goodwill

Mr McNeil said that the Committee’s work ensures that “outputs [of the police’s models] are strictly controlled”. But in reality, much of the Committee’s impact hangs on the police’s and the Police and Crime Commissioner’s goodwill.

Its terms of reference state that the police can, at its own discretion, decide not to consult the Ethics Committee before deploying a new data-based system, if that system is “purely operational”, concerns “extremely serious levels of criminality” or “matters of great urgency”. Committee members are not remunerated for their work and handpicked by the PCC. The Committee has no secretariat and relies on the PCC’s.

In an academic article, Ms Oswald claims that the Ethics Committee does not perform “ethics theater”. On the contrary, it does actual work and contributes to the “trustworthy use of emerging technologies in policing”.

But not everyone is convinced. Griff Ferris, a legal and policy officer at Fair Trials, a non-profit, researched some of the West Midlands police’s projects for the Automating Injustice report. Given the lack of actual legal basis for the Committee’s existence, he expects that, were a conflict to arise, the police’s goodwill would disappear – and with it the impact of the Committee. In particular, he told AlgorithmWatch that NDAS was several years behind schedule and that the police might “get quite frustrated with being pushed back by the Committee”.

Oversight board

When asked whether other police forces should set up ethics committees, European watchdogs were very skeptical. Martin Steiger, a Swiss lawyer who specializes in personal data protection, told AlgorithmWatch that, while Swiss police forces could create such committees, they would not be able to ensure effective supervision.

Martin Drago, a lawyer at the French advocacy group La Quadrature du Net, said that internal mechanisms are poor substitutes for actual, independently-funded control authorities. He likened the Ethics Committee to Facebook’s “Oversight Board”, a group of personalities selected by Facebook that advises the platform and has been lied to by Facebook staff before, according to the Wall Street Journal’s “Facebook Files” investigation.

Ineffectiveness is one of several problems. Benoît Piédallu, a colleague of Mr Drago’s at La Quadrature, said that such advisory committees could undermine other control authorities. By reinforcing the myth of self-regulation, they could end up legitimizing their lack of funding, she said.