POLAND

Automating Society 2019

Introduction EUROPEAN UNION RECOMMENDATIONS ABOUT US

Belgium Denmark Finland France Germany Italy Netherlands Poland Slovenia Spain Sweden United Kingdom

By Alek Tarkowski

Policy debates on the issue of algorithms and automated decision-making (ADM) have only recently been initiated by the public administration, and they have focused on the related concept of Artificial Intelligence (AI). Before 2018, there were no signs of a policy debate on ADM and related issues. However, that changed in spring 2018, when public interest increased.

In April 2018, Poland was one of the Member State signatories of the Declaration of Cooperation on Artificial Intelligence, which was spearheaded by the European Commission. [PL 1] In June 2018, the Polish government announced that work will begin on Poland‘s AI strategy. In November, the Ministry of Digital Affairs published a document titled “Proposition for an AI Strategy in Poland” that includes an action plan for 2018-2019.

ADM solutions are employed to a limited extent by the Polish business sector. According to a report published in 2018 by the Sobieski Institute on “Artificial Intelligence and IoT in Poland” [PL 2], large Polish IT companies do not yet have the capacity to deploy AI or ADM solutions. Greater capacity to do this can be observed among start-ups, micro, and small and medium-sized companies, especially in the FinTech sector. In most cases, these solutions are still in the early phases of implementation and business life cycle. Some of the first ADM projects are also being implemented in the public sector.

There are few visible signs of discussions that concern social, civic or ethical implications of these solutions. By framing the debate around the general term ‘Artificial Intelligence’, Polish stakeholders are avoiding a more specific discussion about the functioning of algorithms and their influence on decision-making. The debate about business solutions that employ ADM is currently focused on a growth paradigm, and explores the potential for further developing this sector of the economy. Similarly, a debate among academics working in the field of Artificial Intelligence focuses on attaining scientific goals, or obtaining public funding either at the Polish or the European level.

Political debates on aspects of automation – Government and Parliament

Polish AI Strategy and Action Plan for 2018-2019

At a conference in June 2018, Jarosław Gowin, Deputy Prime Minister and Minister of Science and Higher Education, declared that Poland will create its own Artificial Intelligence strategy. [PL 3]

In July 2018, the Ministry of Digital Affairs invited stakeholders to participate in shaping the Polish AI strategy. Between August and October, around 200 stakeholders participated in four working groups that dealt with issues of data availability, financing, ethics and regulation, and education in relation to AI. On November 10, 2018, the Ministry of Digital Affairs published a report titled Założenia do strategii AI w Polsce. Plan działań Ministerstwa Cyfryzacji (Proposition for an AI Strategy in Poland. Action Plan of the Ministry of Digital Affairs) [PL 4]. The document presents recommendations of strategic and operational goals for the four areas defined above. It also includes a short action plan proposed by the ministry for the years 2018-2019. It should be noted that the document is not an official strategy—the strategic recommendations have not been in any way endorsed or approved by the government. It remains to be seen if and when the Polish government will present its AI strategy.

The issue of ADM has been extensively addressed by the “Ethics and Law” working group—although the term is not used directly, as the document mainly uses the general concept of AI. The group defines a need to ensure that as AI solutions are being implemented, basic rights are effectively protected. In addition, knowledge about the social impact of AI should be obtained, ethical standards defined, and high-quality regulation adopted for areas related to the implementation of AI. Transparency and explainability of algorithms are listed as one of the key legal challenges concerning the protection of basic human rights in relation to AI.

The proposed 2018-2019 action plan does not include either the implementation of ADM by the public administration or its regulation. The only AI solution proposed in the action plan is a chatbot for the National Qualifications Registry. It is telling that the plan includes little mention of regulatory measures for ADM and AI. Regulation is seen only as a means for providing more effective public support for research, prototyping and implementation of AI in the economy.

The government has declared that work on Poland’s AI strategy will continue in 2019.

Visegrad 4 countries’ thoughts on Artificial Intelligence

In April 2018, the V4 Group (Czech Republic, Hungary, Poland and Slovakia) published a shared position, titled “Visegrad 4 countries’ thoughts on the Artificial Intelligence and maximising its benefits ahead of the release of the European Commission’s Communication on the topic”. [PL 5] The document is rather short and general in its nature. Most attention in the document is paid to the issue of the availability of data, its security, and trust in data sources. The document also declares the importance of “formulating open, executable and recognised international norms concerning the research, development and implementation of ethically designed systems and solutions based on Artificial Intelligence technology”. The document lists the following priorities:

Digitisation as a priority of the European Union beyond 2020
A pan-European initiative on establishing a framework for opening up data for innovation
Opening the debate about a funding mechanism
Uniform regulatory sandboxes
Support for the reform of public administration in decision-making through AI solutions
consideration of cybersecurity and trust issues
examination of the impact on the EU labour force

It should be pointed out that both the regulation of ADM and its use by the public administration are addressed in these priorities. The regulatory sandboxes are understood as “digital, virtual environments, in which interested parties from different sectors can experiment with data and algorithms”. Thanks to these sandboxes, regulatory bodies can observe the development of algorithmic solutions and make informed decisions concerning their regulation. The issue of the use of ADM by the public administration is not further developed, save for the mention of the regulatory impact assessment as a potential process where ADM could be used.

Political debates on aspects of automation – CIVIL SOCIETY AND ACADEMIA

The issue of algorithmic decision-making has been addressed by Polish civil society organisations for several years. In 2018, ADM became a significant topic of public debate due to on-going European copyright reform and the issue of algorithmic content filters. These organisations are also active in the public debate and consultations connected with Poland’s new AI strategy.

Panoptykon Foundation

The Foundation Fundacja Panoptykon [PL 6] was the first to initiate a debate about algorithmic decision-making in Poland. It analysed and then intervened in a case concerning the use of an algorithmic system for profiling the unemployed by the Polish Ministry of Labour and Social Policy (see ADM in Action). The Foundation also addresses the issue of algorithms within its broader anti-surveillance activism.

ePaństwo Foundation

In 2018, Fundacja ePaństwo [PL 7] launched the “alGOVrithms” project, which aims to map the use of ADM by the public administration in Central and Eastern European states. [PL 8] The foundation focuses on the use of algorithms by public administration bodies.

Centrum Cyfrowe Foundation

The Foundation Fundacja Centrum Cyfrowe [PL 9] has been addressing the issue of ADM with regard to content filtering, as part of its advocacy work on the new European Directive on Copyright in the Digital Single Market. The issue, dubbed “ACTA 2” (in reference to the infamous ACTA treaty, which sparked mass protests in Poland), became one of the key digital policy issues in the Polish public debate this year, alongside GDPR implementation. In autumn, the Foundation launched a public campaign on the issue, called “Internet is for the people”. [PL 10]

Coalition for Polish Innovation and Digital Poland

By 2018, two major cross-sector coalitions, Koalicja na rzecz Polskich Innowacji (Coalition for Polish Innovation¹) and Digital Poland², launched working groups and a programme that dealt with issues related to AI and algorithms. Both of these coalitions are foundations, established with the goal of networking stakeholders (mainly business, but also civil society or academic institutions) on issues related to the regulation of digital technologies. These groups were largely formed in response to the growing interest of the government in Artificial Intelligence and related issues. However, at the time of writing this report, neither of the two coalitions have published any position papers or recommendations on the issue.

Regulatory and self-regulatory Measures

At the moment Poland lacks any regulation on a national level that directly concerns algorithms or algorithmic decision-making.³ Although several important case studies of the use of ADM can be observed at different levels of government, there doesn’t seem to be any attempt to regulate this issue or to define standards. Neither exist programmes to raise awareness about the issue or to encourage implementation of ADM solutions. The solutions that are employed are also relatively simple algorithmic processes.

ADM in action

Ministry of Justice – System of random Allocation of Cases

On 1 January 2018, the Polish Ministry of Justice introduced the “System of Random Allocation of Cases” (System Losowego Przydziału Spraw), a digital system that, on a once-per-day basis, assigns cases to judges across the country. The system is currently the most visible and most discussed case of ADM used by the public administration.

Pilots of the programme were initiated in three cities in 2017. The launch of the system aligned with controversial reforms of the judicial system in Poland, leading to intense public scrutiny. According to anecdotal evidence, the allocation of cases was not random, with judges receiving extremely varied allocations, often seen by them as unfair. In particular, the random character of the algorithm has been questioned.

In 2017, the Ministry refused a freedom of information request by NGOs ePaństwo and Watchdog Polska to disclose the source code and the details of the algorithm that powers the system. [PL 12]

The Ministry has provided explanations on how the system functions and described how the algorithm works, but it refuses to make the source code publicly available. According to the ministry, system administrators have limited permissions and cannot interfere with the randomised selection, the working of the programme is overseen by the Appellate Court in Gdańsk, and all operations made by users are registered. The Minister of Justice has declared that "the selection will be made solely by a machine, a computer system that is blind like Themis, and chooses without emotions, without views or biases, and in a manner fully free from possible accusations of corruption“. [PL 13]

In mid-2018, the Ministry admitted that the system has faults and, in the case of some judges, assigns cases unequally. The ministry promised to introduce changes to the algorithm, yet its exact functioning remains unclear and controversial. [PL 14]

School systems – allocation of children at schools

In Wrocław, one of Poland’s largest cities, a system malfunction caused the algorithm that allocates children to nurseries to make incorrect selections. The malfunction led to a rare case of public scrutiny of the use of ADM for allocating learners to nurseries, preschools and schools. [PL 15]

In a typical scenario, a citywide allocation system is used to place children in preschools. This is done based on information provided by parents, which can include data on such factors as the number of children, single-parent households, food allergies of children, handicaps, and material situation. The functioning of such systems is controversial especially at preschool level, where the lack of a sufficient number of preschools and the lack of any obligation for a child to be in preschool leads to a shortage of available places. Parents therefore feel subjected to an arbitrary system that allocates their children in a non-transparent and possibly unfair manner.

There is no data available on the scale at which such systems are employed. One of these systems, Platforma Zarządzania Oświatą (Education Management Platform) created by Asseco Data Systems [PL 16], is used by 4,500 schools and preschools in 20 Polish cities. The system offers a range of functions and there is no data available on what percentage of them use ADM solutions for the allocation of learners. These cases of ADM do not seem to draw much public scrutiny. However, anecdotal evidence shows that the decisions made by these systems are of great importance to the parents of children in preschools and schools.

Canard Speed Camera System

The Centrum Automatycznego Nadzoru nad Ruchem Drogowym (Road Traffic Automation Supervision Centre, CANARD) [PL 17] is a nationwide fotoradar (speed camera) system. It is connected to an IT system that uses image analysis algorithms to read license plates before it automatically fines drivers who are speeding. The centre was created in 2011 within the General Road Transport Inspection (Główny Inspektorat Ruchu Drogowego), and the traffic control infrastructure that uses ADM was launched in 2015. The system consists of 400 stationary speed cameras, 29 mobile units, 29 road-based traffic measurement systems, and 20 devices that register vehicles crossing intersections at a red light. In August 2018, the “Puls Biznesu” economic journal disclosed that the CANARD system fails to fine drivers of electric cars. [PL 18] Apparently, incomplete data on electric cars in the CEPiK national vehicle registry led the algorithm to identify electric cars as belonging to a special category (reserved, for example, for traffic control vehicles) that were exempt from speeding fines.

Ministry of Labour and Social Policy – profiling of unemployed people

In May 2014, the Ministry of Labour and Social Policy introduced an ADM system that profiles unemployed people and assigns them to three categories that determine the type of assistance that a person can obtain from local labour offices. [PL 19] The profiling mechanism is part of “Syriusz Std”—a nationwide IT system created by the IT department of the Ministry—which collects data on people who register as unemployed in labour offices, and on employees and their activities. The decision is made based on data collected through a questionnaire used by an employee of the labour office to question the unemployed person. 24 questions are used to collect data on two criteria that are taken into account: “distance from the labour market” and “readiness to enter or return to the labour market”.

Fundacja Panoptykon, and other NGOs critical of the system, believe that the questionnaire, and the system that makes decisions based on it, profiles individuals based on personal data. Once the system makes a decision based on the data, the employee of the labour office can change the profile selection before approving the decision and ending the process. According to official data, employees modify the system’s selection in less than 1% of cases. Furthermore, the system has been criticised for a lack of transparency about the distribution of public services, lack of oversight, and the alleged arbitrary nature of decision-making due to the simplification of data obtained from interviews. In addition, the system does not give the subjects of this ADM process the means to obtain data or an explanation concerning the decision, and the labour offices have limited means of analysing and evaluating the ADM process. Initially, the Ministry shared general information about the functioning of the algorithm with the Panoptykon Foundation, which investigated the case and which later made this information public. In 2016, the Foundation obtained detailed information about the questionnaire and the scoring algorithm through a freedom of information request. [PL 20] In the same year, the Polish Commissioner for Human Rights asked the Polish Constitutional Tribunal to determine whether the profiling system is in line with the Polish constitution. However, the issues raised by the commissioner did not concern the ADM component, instead he questioned the lack of a redress mechanism and the basis for collecting personal data. [PL 21] The case was solved in 2018, when the Constitutional Tribunal decided that the system needs to be better secured in a legislative act. [PL 22]

National Health Fund and fraud detection

In September 2018, the National Health Fund (NFZ) announced that it will implement algorithms to enable closer scrutiny of public health expenditure. [PL 23] Currently, regional NFZ offices use two different IT systems that are not integrated, and thus do not enable full data analysis. Data integration is therefore the first step of the project announced by the fund. Using the data, algorithms will compare an individual patient’s history of medical procedures with standardised scenarios, in order to discover anomalies—which are potentially caused by fraud at health institutions. The NFZ estimates that by employing algorithms it will shorten the time needed to analyse all contracted health institutions from 16 years to 5 years.

One2Tribe

One2Tribe [PL 24] is a Polish company that started out as a games developer, and then pivoted into providing a motivational platform based on gamification methods and behavioural psychology. In recent years, the company has been developing a machine learning solution that optimises the motivational platform to most effectively improve employee behaviour by appropriately defining challenges and prizes. Recently, they have also begun implementing algorithms that take into account individual work stress as a factor. In all cases, ADM is being used to tailor the system to the individual traits and preferences of employees. Their new venture, one2tribe labs, will use the same platform to motivate patients undergoing medical therapy. [PL 25]

Nethone

Nethone [PL 26] is a Polish company that works on ADM solutions for detecting financial fraud and has developed what it calls a “Know Your User” (KYU) solution (based on the concept of “Know Your Customer”, which is a cornerstone of most financial products). The company makes predictions based on user-website interaction, hardware specifications, and other data points provided by their business partners. Machine learning solutions developed by Nethone are used to verify the identity of individuals making online transactions, in an effort to identify fraudulent transactions. The company is currently active mainly in Latin American, UK and US markets. In 2018, Nethone received a grant from the Polish National Centre for Research and Development to develop an ADM solution for combatting account takeover (ATO) attacks on bank account holders. The solution could in principle replace current authentication and monitoring methods. In September 2018, Nethone partnered with the Polish Association of Lending Institutions (PZIP), which gathers together more than 50 Polish institutions providing non-bank loans. The association recommends the Nethone system to its members as an additional anti-fraud solution.

1 Website of the coalition: https://koalicjadlainnowacji.pl/en/

2 Website of the foundation: https://www.digitalpoland.org/en/

3 This opinion is shared by the authors of the report: Polityka Insight, “Iloraz sztucznej inteligencji. Potencjał AI w polskiej gospodarce”, 2018 [PL 11]