SLOVENIA

Automating Society 2019

Introduction EUROPEAN UNION RECOMMENDATIONS ABOUT US

Belgium Denmark Finland France Germany Italy Netherlands Poland Slovenia Spain Sweden United Kingdom

By Lenart J. Kučić

Slovenia has not adopted any strategic document on a national level regarding the use of Artificial Intelligence (AI), algorithms or automated decision-making (ADM) systems. However, in April 2008, the Ministry of Public Administration invited [SI 1] all stakeholders in the field of AI to help them find use cases and outline future needs / challenges for both the public and private sector. According to the ministry, they will ask the government to form a working group that will outline a national strategy in the first half of 2019.

Slovenia also signed the European “Digital Day” declaration to “encourage cooperation in artificial intelligence, blockchain, eHealth and innovation.” [SI 2] [SI 3]

Political debates on aspects of automation – Government and Parliament

Digital transformation and eGovernment

The Ministry of Public Administration issued two publications presenting Slovenian e-government’s services and its “reference vision” for digital transformation. The first publication eGovernment in Slovenia describes government databases and services. [SI 4] The second—Digital Transformation of Slovenia [SI 5]—also mentions Artificial Intelligence and information infrastructure for future automated systems (e.g. driving, social security).

The Ministry for Public Administration is keeping most of the government’s and citizens’ data. According to a spokesperson, they have not yet tested or implemented any kind of autonomous systems but are using machine learning to improve their internal processes, e.g. the efficiency of government services and coordination of databases.

Political debates on aspects of automation – Civil Society and Academia

Financing Hate Speech

Slovenian online activist Domen Savič started a campaign documenting online ads from big Slovenian advertisers which appeared on various tabloid, political, astroturf, and disinformation websites. [SI 6] He argued that advertisers are financing hate speech and called on companies to stop advertising on such websites. Most advertisers refused to comment. Some have tweaked their ad placement practices and silently removed their ads from problematic websites. One company admitted—or rather blamed—that “advertising programs” are responsible for ad placement and denied any responsibility for the placement of their ads.

In particular, Domen Savič focused on Telekom Slovenije—a majority state-owned national telecommunications company—because they were running ads on the website Nova24TV [SI 7], which was founded by the biggest Slovenian centre-right political party SDS. Savic noted that Telekom Slovenije presented itself as a responsible company promoting “respect for all individuals”. [SI 8] Nonetheless, they decided to advertise on Nova24TV which regularly featured problematic, hateful, and false articles on migrants, ethnic and religious minorities (e.g. Muslims).

Telekom Slovenije stated that they do not promote any kind of intolerance with their advertising. Their only goal is to present their services to as many potential customers as possible. They also claimed that they do not want to interfere with editorial policy in any way.

The president of the Slovenian Advertising Chamber [SI 9] explained that advertisers have to balance their corporate responsibilities and brand values on the one hand with editorial independence on the other. However, the Slovenian Advertising Chamber supports the idea of a “Code of Practice on Disinformation”—an initiative of European advertisers [SI 10] proposed to the European Commission in September 2018. The Code of Practice calls for more transparent media buying practices and recognises the possibility that advertisers may involuntarily finance disinformation websites.

The two co-founders of the Slovenian content marketing company Zemanta [SI 11] (which was recently taken over by the US company Outbrain) explained the functioning of the advertising “black box” and advertising algorithms. They admitted that advertising algorithms can sometimes be gamed and abused by disinformation sites. However, they did not agree with the argument that autonomous advertising mechanisms can be held responsible for financing hate speech. Indeed, the advertiser may be surprised by a certain “unfortunate ad placement”. But they argue that it is relatively easy to fix this problem if an advertiser decides to better control its online ads. In short: “they (the advertisers) should learn to better use their tools”.

Slovenian Prime Minister Marjan Šarec also joined the debate in November 2018. He published a public statement [SI 12] and called for the advertisers to reconsider their marketing strategies when their clients promote hate speech (e.g. racism and homophobia). His appeal evoked radically polarised opinions. Political opponents, some advertisers, and right-wing and conservative media commentators accused him of censorship and political pressure on private companies. Left-wing and liberal media critics, on the other hand, embraced the PM's position that advertisers should not legitimise and finance hate speech.

Petition against automated weapons

In October 2018, a group of scientists, legal and technological experts organised a public debate on AI, autonomous weapons, and the risks of intelligent machines taking over the government. [SI 13] They produced a joint statement and petition against research and the use of autonomous weapons. Such weapons should be banned in a way similar to chemical weapons, cluster bombs, and laser systems for blinding human combatants. The group also appealed to the Slovenian government to join the UN coalition against such weapons and support the recommendation of the resolution of autonomous weapons proposed by the European Parliament. [SI 14]

#SOCRATECH – responsible IT development

In September and October 2018, the Slovenian online activist Domen Savič (he is a co-founder of an NGO called Državljan D—“Citizen D”) organised an international tour of panel discussions and public talks about ethical uses of IT. [SI 15] The tour included Slovenia, Croatia, Serbia, and Bosnia.

Speakers discussed the following topics:

What is the meaning of ethical IT development?
Why are smart cities dangerous?
Why do we need to talk about Big Data?
How can we protect ourselves against aggressive Artificial Intelligence and
smart algorithms?

Regulatory and self-regulatory Measures

SI ST

The technical secretary at the Slovenian Institute for Standardisation (SI ST) [SI 16] said that they have not yet received an application for standardisation of AI. However, they have registered the following standards in the field of autonomous systems / algorithms:

SI ST ISO 9564-2:1995—Banking; managing and protection of personal identification numbers (PIN)
SI ST-TP ETSI /SR 002 176 V1.1.1.2005—Electronic signature and infrastructure (ESI )
SI ST ISO 10126-2:1995—Banking; encryption algorithms (DEA)
SI ST EN ISO 12813:2016—Electronic collection of fees / communication for verification of compliance of autonomous systems (ISO 12813:2015)
SI ST-TS CEN/TS 16702-1:2015, SI ST-TS CEN/TS 16702-2:2015—Electronic collection of fees / secure monitoring of autonomous tolling systems

ADM in Action

Airport (border) police

On November 2, 2018, Slovenian daily Delo published a story about an Albanian citizen who tried to enter the Schengen area at Jože Pučnik international airport. [SI 17] A “special algorithm” warned a border police officer to make additional checks on the suspect. The police found some falsified documents, Western Union money transfers and photographs for acquiring a false visa.

The Delo newspaper report stated that the police have acquired information about almost 800,000 airline passengers (so-called Passenger Name Records, PNR ) since October 2017. The system tagged 8,928 passengers who were then thoroughly inspected before entering the country. The police stated that 40 per cent of those passengers were tagged as “not suspicious” and will not be reported next time they come to the border. Airline companies provided the data.

A police spokesperson explained they are not using algorithmic decision-making systems in this process. The system automatically matches a passenger to “other police data” such as criminal files. If a match is positive, a border police officer is informed and required to manually verify the information before inspecting a passenger. The police system also flags passengers who are using “unusual or illogical flights”.

The Slovenian Human Rights Ombudsman and the Information Commissioner stated that such a system is not constitutional and filed a formal complaint in 2017. [SI 18] [SI 19] The Information Commissioner claimed that the adopted changes of the amended law on the duties and powers of the police [SI 20], which gave the police the power to gather the PNR, have legalised some excessive and inadmissible measures for gathering personal data without sufficient protection of citizens that have not been accused or suspected of any wrongdoings, e.g. terrorism or organised crime. They argued that all passengers should not be routinely scanned at the airport just because they are entering the state or landing during the transfer. The Human Rights Ombudsman supported their claims and the Slovenian Constitutional Court will therefore be required to rule on the constitutionality of the latest revision of the law on the duties and powers of the police.

Banking

The largest Slovenian bank NLB uses algorithmic systems in their operations to support decisions on granting loans and mortgages, to assist with fraud detection, and to improve the quality of their data (preventing wrong inputs and typos from clerks etc.). NLB’s Chief Data Officer explained that their “quick-loan” offers (available via the mobile banking app) are almost entirely automated and only require final confirmation by a human. [SI 21] All Slovenian banks are required to access SI SBON—an information system for the exchange of data on individual debts before granting a loan. [SI 22] SI SBON is operated by the Bank of Slovenia and includes personal data and credit operation of individuals (credit data is held for four years). SI SBON has to be accessed by a person, not an automated system, according to NLB. Legislation thus limits the possibility for developing fully automated banking services for the time being. [SI 23]

NLB also said they do not discriminate against customers based on their demographics. “Slovenia is still a relatively egalitarian society and it does not make much sense to profile individual customers on their gender, place of residence, ethnicity or other categories,” according to NLB’s Data Officer.

METIS – detecting learning problems in primary and secondary schools

METIS is an “intelligent system for early detection of learning problems in primary schools.” [SI 24] It is a joint project of the Jožef Stefan Institute and the Slovenian ministry of Education, Science, and Sport. [SI 25] The system has the potential to monitor pupils’ grades (and absences) in 72% of all Slovenian primary and 90% of secondary schools. It will use machine learning to search for specific learning patterns and help teachers find “problematic” pupils. The researchers have built a detection model based on “more than 30 million grades” and “some other data”. However, “teachers will keep all autonomy”, according to spokespeople at the press conference. [SI 26] [SI 27]

METIS was first introduced in the summer of 2017. However, the representatives of the ministry of Education, Science, and Sport did not provide any additional information on when (or whether) the system will be fully implemented. According to a representative from the Jožef Stefan Institute, schools were required to obtain written consent from parents in order to run the system. This became the biggest obstacle for its adoption: “It was too complicated”, the representative said.

Insurance

The biggest Slovenian insurance company Triglav [SI 28] uses algorithmic systems to help their insurance agents recommend their insurance products to customers, to detect fraud, and to assess insurance risks. Nevertheless, they only use algorithmic tools as counsellors—they leave the final decision to their human agents, according to a spokesperson.

Tax-evasion and tax-fraud

The Ministry of Finance’s financial administration [SI 29] confirmed that they are using machine learning to detect tax-evasion schemes and tax fraud, and to find errors in tax reports among other uses. They complement algorithmic models with real-life information from tax inspectors to better “teach” the algorithms. They are also developing mathematical tools for data mining and prediction analysis to find future improvements of the tax system (optimising, modifying, and collecting taxes).

A spokesperson from the financial administration confirmed they are ranking “risky” citizens who are more likely to become tax evaders. Their ranking depends on the type of a tax. A taxpayer who is likely to evade an income tax may not be the same person as someone who will try to evade a value added tax.