This story is part of AlgorithmWatch's upcoming report Automating Society 2020, to be published later this year. Subscribe to our newsletter to be alerted when the report is out.
When Slovenian journalists or activists ask officials whether the police, the secret service, and the army are using any technological tools for mass surveillance, they are often reluctant to admit even the existence of such devices. However, when they meet their colleagues at international security conferences or receive questions from foreign journalists, they want to make a positive impression. “Of course we are testing (or using) the latest new gadgets to fight terrorism, organized crime or illegal migration, they say. We are a modern and technologically advanced security force after all!”
This may sound anecdotal but it happens to be true.
When Slovenian civil society activists and privacy experts read the 2019 AlgorithmWatch report on face recognition in the EU, they were surprised to find Slovenia listed among the countries where the police use face recognition.
“Slovenian police personnel have been using face recognition since 2014. The software has been developed internally,” the report claimed, quoting the Slovenian Police as the source.
It turned out Slovenian journalists and privacy activists might have been following the wrong sources.
How do you detect evil aliens before they can invade human society? You need to follow the “hot sheets”, said a special agent “K”, played by American actor Tommy Lee Jones in the hit sci-fi movie “Men in Black”.
What he meant by “hot sheets” were the supermarket tabloid magazines reporting on alien abductions and other strange and bizarre stories. You may also get lucky by reading the New York Times, “K” explained to the rookie agent “J” (Will Smith). But tabloids are a much better source of information if you want to control alien activity on Earth and protect humanity.
But the “hot sheets” do not only reveal potential aliens. They can also alert privacy advocates to unusual police activity.
“Aggressive pensioners recognized on Facebook” claimed the headline in Slovenske novice, the biggest Slovenian print tabloid. A woman from Ljubljana, the Slovenian capital, left her car on a bicycle track. She left with the blinkers on, and she ran to a nearby restaurant. It only took her a minute to come back, the tabloid wrote, but her parked car had already annoyed an elderly couple.
The elderly man smashed one of the side mirrors with his fist and then tried to hit the woman as well. She took a photo of the couple with her phone and reported the attack to the police. She also published the photos on Facebook where some users said they recognized the violent couple.
The tabloid article mentioned that the police had been using a new piece of face-recognition software called Face Trace to find suspects using open source investigation methods (such as searching social media and other online sources).
The tabloid article was originally published in December 2015 – four years ahead of the AlgorithmWatch report on face recognition. The CEO of the company that developed the Face Trace software even linked the piece to his social media account. After a thorough research of the Slovenian media archives for this feature, I found one even earlier mention of Face Trace in one of the major Slovenian daily newspapers, Delo, where a journalist talked to the police operator who was responsible for face recognition in 2014.
But nobody among Slovenian civil society took notice at the time.
“I saw my friend in the police database”
The articles in Slovenske novice and Delo about face recognition were not isolated examples.
In January 2017 a woman contacted a journalist writing for the national television website MMC. She was stopped by the traffic police. She did not carry her ID and the police officer brought her to his car to confirm her identity on the police computer. She gave her personal information (name, address, date of birth) and her photos appeared on the screen. She expected to see her official photos – from the ID, passport, and the driving license – but she also noticed photos that she only published on her personal Facebook profile.
Why would the police store her Facebook photos in their database, she wondered. Was that even legal?
The police declined to confirm or deny that they gathered pictures from social media profiles in their database of photographed persons. Their spokesperson then suggested to the journalist that the police officer may have also checked publicly available photos on the internet during the procedure, which could have given a wrong impression to the driver (that her Facebook photos were stored in the police database). Instead, they claimed that everybody could access the information on social media and provided a lengthy legal argument on why a citizen must always carry a valid ID to avoid such police procedures and paying a fine.
Another similar example dates back even further – before Face Trace was officially introduced. A citizen told us that they filed a complaint to the Information Commissioner in September 2013. The complaint includes a passage where a young woman reported an attempted sexual assault to the police. When she browsed through the police database to find and recognize the alleged attacker, she also saw a photo of her friend.
“He was never accused or convicted of any wrongdoings,” reads the complaint. He did join some peaceful public protests, though. Is it possible that the police takes photos of the protesters and stores them in the database of photographed individuals, or in the database of “extremists”, she wondered. And on what legal ground could the police do such a thing?
The Information Commissioner officially replied to the complaint in February 2014. The police denied that a database of “extremist” protesters existed, the Commissioner wrote. But they explained that the database of photographed individuals could also contain a photo of one’s friend or relative if this person has ever been suspected of a criminal act. The police can also record or photograph public events and keep “relevant pieces” of the material. At that time, the police database of photographed individuals contained around 30,000 persons, according to the Commissioner.
From the examples above, it is possible to conclude that Slovenian police has been using a face recognition software since 2014. The police also confirmed to me that they use face recognition systems to compare, for example, police sketches to pictures in their database of photographed individuals. However, it is likely that faces from the police database of photographed individuals are used, too. This is important because the database could contain photos from personal social profiles and mass public events.
We are using conditionals because we do not know for certain what photographs the police actually keep in their database. Official answers provided by the police remained ambiguous during our investigation and the Information Commissioner has not yet decided to investigate the matter and evaluate the use and content of police databases (such an inspection is scheduled for this year, according to the Commissioner).
Hacking the law
In addition, the police is often allowed to interpret the legislation as it best suits their needs – without much opposition from the public, politicians, or privacy experts. “The police always follows the same pattern. First, their representatives dramatically demonstrate all the new threats to the public life: international terrorism and organized crime, migrant crisis, and online threats – from pedophilia to cyber attacks. Next, they express their frustrations and concerns because they cannot efficiently fight the bad guys who have all the latest tools and no legal restrictions on how to use them. So they wait for every opportunity to amend the existing legislation and expand their powers,” explained a public official who took part in several negotiation rounds with the police and requested anonymity.
The police always presents a very ambitious wish list, our source said. Some claims are clearly not realistic and would never pass the legislative process: weakening the encryption on communication apps or using drones for mass surveillance. But they can be used in negotiations with the Information Commissioner and other privacy advocates. Fine, let us not touch the encryption, says the police. But we still need to amend the outdated law on duties and powers of the police and include, for example, biometrics. After biometrics is mentioned in the legislation the police can interpret this as a legal basis for using face recognition.
“They are hacking the legislation this way. This is like installing a piece of malware to the operating system and then using the exploits to legalize whatever you are doing,” concluded the source.
Such hacking of the legislative process by the police is not merely hypothetical. When the Slovenian police bought its first IMSI catcher in 2004, a tool to snoop on cell phones, there was no legal basis for its use, the Slovenian online medium Slo-Tech revealed. Nevertheless, the police secretly used the machine in more than 300 cases, from 2006 to 2012, and then bought another IMSI catcher in 2012.
When lawyers, journalists, and activists first challenged the allegedly illegal use of the IMSI catchers the police spokesperson denied the existence of such a device. However, they had to change their communications strategy after a high police official said they were using IMSI catchers “to find missing persons” in an interview for a public radio. Instead of denying their purchase and use, the police tried – with a lot of help from the government – to retroactively legalize IMSI catchers.
The police approach to face recognition was very similar.
The Slovenian Information Commissioner confirmed that the police informed them about the use of face recognition software Face Trace back in 2014. The police claimed that the Commissioner agreed to their suggested uses of the product. But the Commissioner’s office actually issued several critical comments regarding the use of biometric methods and face recognition between 2015 and 2019. They also appealed to the Slovenian Human Rights ombudsman to file a formal complaint to the constitutional court when the Slovenian Ministry for Internal Affairs introduced a new law on the duties and powers of the police in 2017.
The new amendments allowed the police to use biometric and face recognition tools, and thus provided a legal framework for the use of the Face Trace software. Again, the police tried to retroactively legalize its use and the government ignored all the warnings from the Information Commissioner. How was this possible? There was no public pressure, argued the Information Commissioner. Nobody seemed to care about their warnings against potential police abuse of biometry and face recognition. As a consequence, the police is now allowed to automatically combine face recognition with other biometric data like fingerprints and DNA profiles.
No public pressure?
The police, nonetheless, did face some opposition. In the 2019 Automating Society report, we learned that the Slovenian Human Rights ombudsman, and the Information Commissioner, filed a formal complaint to the constitutional court in 2017. They claimed that the new law on the duties and powers of the police legalized some excessive and inadmissible measures for gathering personal data without sufficient protection of citizens who have not been accused or suspected of any wrongdoings. In addition, they warned against the use of drones and automated systems for recognizing license plates, as well as unregulated use of passenger name records at Slovenian airports. But for some reason, biometric systems (such as face recognition) were not included in the complaint.
How so? The Information Commissioner explained that they suggested to the Slovenian Human Rights ombudsman that biometrics should also be included in the complaint. But that did not happen. The representatives of the ombudsman explained (to us) that there had been no public debate or controversy surrounding the use of biometrics by the police at the time. They had a legal option to file another complaint to the constitutional court and address the use of face recognition but they did not take any action. Why? Because they did not want to over-use this exceptional legal measure, according to the ombudsman.
This argument clearly illustrates some of the problematic aspects regarding the implementation and use of Artificial Intelligence (AI), algorithms and automated decision-making systems (ADM) in Slovenia. During our research for the previous Automating Society report, very few civil society organizations or individuals addressed the social impacts of automation on society. In addition, conferences about AI are usually sponsored and organized by commercial companies and industry groups who want to promote their solutions. Furthermore, many of the specialized journalists who cover such issues have quit the media in recent years. As a result, there is no rigorous coverage on the topic and privacy experts or activists have lost access to the general public.
Civil society should not be blamed for this state of affairs. “Slovenian non-governmental organizations (NGO) are small and understaffed. As far as I know, no NGO is monitoring the police and the use of their powers systematically,” said Katarina Bervar Sternad from the Legal Centre for the Protection of Human Rights and Environment. “We cannot afford to dedicate one staffer to this particular field. This is unfortunate because we receive many questions regarding potential abuses of police powers.”
Domen Savič from the NGO Državljan D makes a similar point. “The NGO sector is mostly funded by the government or international EU grants,” Savič said. If there are no tenders on information society, NGO cannot afford to cover this field. Furthermore, the media and politicians rarely understand how technology impacts human rights. “When AlgorithmWatch revealed that Slovenian police was using face recognition software to profile its own citizens and some Slovenian media investigated the story, there was not a single voice of protest raised in the parliament.”
To make things worse, there is almost no collaboration between politics, the media, and civil society, warned Savič. “It takes an extraordinary amount of effort, time, and pressure to actually change a policy or a police practice. But most of the issues die out pretty quickly even if there is someone who tries to get answers from the people involved.”
Consequently, the police faces almost no opposition when handing in their “wish-list” to the government.
Another worrying example is a recent attempt by the Ministry of Justice to use the GDPR - to which the necessary amendments to Slovenian legislation must comply - to exclude the police from civilian control. The Information Commissioner warned that the ministry wanted to redefine its role under the new (amended) law on personal data protection. According to the ministry’s proposal, the Information Commissioner would no longer be able to oversee the police.
Such safety mechanisms are critical and monitoring of the police should not be left only to NGOs and activists. Mr Savič, Ms Bervar Sternad and the unnamed official all agreed that a common European regulatory approach would be necessary because national NGOs cannot effectively monitor how the police uses the technology. Member States, on the other hand, can use the sense of emergency to adopt new pieces of legislation and bypass parliamentary debate. The current health crisis is the most recent example of such an attempt.
Soon after the pandemic of Covid-19 broke, earlier this year, the Slovenian government started using the crisis as an excuse to expand police powers significantly.
The method was similar to previous attempts of legislation hacking. In March 2020, the Slovenian government proposed a first draft of the “Act on the intervention measures to mitigate the consequences of the communicable disease SARS-CoV-2 (Covid-19) for citizens and the economy”. They submitted the text to the National Assembly for consideration and adoption under an emergency procedure. However, the draft also included two articles that dramatically increased the powers of the police.
Article 103 suggested that the police could use various methods to ensure that the citizens obey the quarantine and the Communicable Diseases Act. They could – among other measures – use face recognition when stopping and identifying individuals, enter their houses or apartments, limit their movement as well as collect and process protected personal information such as medical data provided by the National Institute of Public Health. Article 104 went even further. It suggested that the police could trace the location of mobile phones of individuals without a court warrant.
All the suggested measures were introduced under an emergency procedure – without any consultations or public debates. The Information Commissioner thus described the anti-corona measures as an attempt to potentially “establish a police state”. They considered the new police powers to be too broad and potentially unconstitutional and undemocratic. The Human Rights ombudsman wrote that it was hard to believe that such measures were really necessary and proportional (neither institutions were consulted during the process). A critical commentary was also published by the members of the Institute of Criminology who wrote that mass surveillance is not compatible with European legal culture.
Article 104 was eventually removed from the amended act because of strong criticisms from the public and the opposition political parties. However, article 103 on the powers of the police remained in the “Covid-act” that was adopted in April 2020. Furthermore, the government insisted that contact tracing applications were necessary to help health officials stop the pandemic. They also suggested that citizens would have to install such an application when traveling across the country. Data from the application would be collected and used by the National Institute of Public Health but the police would also be allowed to access the database.
When the first groups of citizens started protesting against the government in April 2020, Interior minister Aleš Hojs demanded on Twitter that the police use their new powers to identify and prosecute the protesters, such as collecting and analyzing all available images from traditional and social media. The government claimed that the protesters were violating laws on public health, and that any public gathering was illegal.
According to what we know, the police has the ability to collect and analyze photographs from social media. The police also routinely records public events, protests, and other mass gatherings. Furthermore, the new legislation could allow the police to access other kinds of personal information (contact tracing) to curb future anti-government protests.
It takes many small steps to build a – partly automated – surveillance infrastructure. The Slovenian police has been consistently taking such steps and the “Covid Act” has most likely legalized most of the items on their “wish-list”, from face recognition to contact tracing.
Did you like this story?
Every two weeks, our newsletter Automated Society delves into the unreported ways automated systems affect society and the world around you. Subscribe now to receive the next issue in your inbox!