DENMARK

Automating Society 2019

Introduction EUROPEAN UNION RECOMMENDATIONS ABOUT US

Belgium Denmark Finland France Germany Italy Netherlands Poland Slovenia Spain Sweden United Kingdom

By Brigitte Alfter

The Danish government states that it wants to actively further the development of Artificial Intelligence (AI) and related education. The focus is clearly on the potential for economic growth. Activities include support for digital qualifications in general education, funding for research and support for business innovation. A government commission on data ethics has recommended labelling products and services that contain AI-based technology, and it also suggests the creation of a permanent data ethics council. Digital tools, AI, and automated decision-making (ADM) systems are being integrated into public administration processes. Many such ADM systems are not discussed by the wider public and are simply considered to be efficient administration. But some cases have led to widespread political debate, such as a surveillance and early warning system for children in vulnerable circumstances. Specialists in civil society and academia call for transparency, accountability, and adjusted legislation that balances both digital efficiency and civil liberties.

Political debates on aspects of automation – Government and Parliament

Digitalisation reform of the public sector

On October 23 2018, the digitalisation reform of the public sector was announced. Its purpose was to better use citizens’ data, and to use AI to make public administration more efficient and citizen-friendly. [DK 1]; The plan includes a strategy for the public sector, plans for a data ethics council, and the ability for each citizen to see all the data held about him or her, including a log about who accessed this data and when. The plan also points towards providing private companies—such as insurance and banks—access to public data, including citizens’ data, and thus support the national strategy for digital growth passed earlier the same year. The plan is supported by an investment fund of 410 million Danish Crowns (€55 million) for 2018-2022. [DK 2]Denmark is rather unique in that, since the 1960s, data compiled on citizens has been recorded with a unique identifier, a personal number. The government hopes to use this data when developing Artificial Intelligence based on large data sets. [DK 3]Though the term automatisation is only mentioned in connection to ‘routine’ tasks, one of the explicit purposes of the plan is to use Artificial Intelligence to better service citizens in areas such as medical prediction systems, or better control of fraud. The plan is set to be implemented over the coming years.

National strategy for digital growth

A national strategy for Denmark’s digital growth [DK 4] was published in January 2018 with the overall purpose of stimulating growth. The logic is business driven: that digitalisation leads to improved productivity per worker, that about one third of jobs in Denmark can potentially be automated, and that digital developments create new competition for Danish business. The Danish government and two other political parties have allocated 1 billion Danish Crowns (€134 million) from November 2017 until 2025 for stimulation and development efforts. In addition, certain tax incentives are offered for new initiatives and development in the field. The activities include a Digital Hub [DK 5] set up in a public-private-partnership by three ministries and three business organisations: the chamber of commerce, the industry association, and the financial sector. The purpose is to help matchmaking between specialised research, competence and investment, to do further research and to market Denmark as a digitally attractive business environment. Other elements are: a Technology Pact [DK 6] that allocates resources to digital education in schools (this is the single largest sum budgeted) and includes coding for teachers and children or case-oriented projects in collaboration between schools and local businesses [DK 7]; a focus on the use of data as a growth driver, including open public data and public-private data sharing; revision of existing regulation to make it easier for business to develop and use new technologies; a catalogue of legislation to be adjusted to the needs of business and consumers and the strengthening of IT security in business. This strategy has to be considered within the context of automated decision-making, even though the strategy does not explicitly mention ADM.

Strategy on digital health

In January 2018, Denmark passed a strategy on digital health for 2018-2022 [DK 8].The plan is to develop a number of digital or digitally supported services, including automatisation, prediction and ‘decision support’.

The government’s ethics council, Etisk Råd, advocates a balanced approach with a particular focus on privacy rights. [DK 9] Doctors’ organisations are generally in favour of digital developments, though they raise the flag when they see the physician-patient privilege threatened, or when terminology lacks legal clarity. On a more general level, digital developments—including in the health sector and particularly when it comes to (automated) predictions—are addressed by data ethic advocates/consultants, [DK 10] who, for example, point out the dangers of insecurity related to automated predictions.

Government strategy for research and education

The government strategy for research and education from December 2017 [DK 11] emphasises the use of digital technologies. This covers both the development of entirely new technologies and the application of digital technologies in business and in the public sector. Among other initiatives, the government is also working towards a national centre for digital technologies.

Public authorities’ digitalisation strategy

In May 2016, public authorities at the national, regional and local level—including administrative bodies such as ministries as well as implementing bodies like public hospitals, schools etc.—decided on a digitalisation strategy 2016-2020. [DK 12] However, this strategy did not explicitly include automated decision-making.

Political debates on aspects of automation – Civil Society and Academia

A number of fora in Denmark discuss digital growth, opportunities, needs and ethics. Most of them address this on a general level, however some address automated decision-making in particular.

Siri-Commission

The Siri-Commission was initiated by a social liberal politician and the union of engineers. Its leading group [DK 13] consists of high level trade union and business representatives predominantly. Its purpose [DK 14] is to look into growth and job opportunities connected to Artificial Intelligence and to raise awareness of the effects of such changes on Danish society. In September 2011, the Siri Commission published a report prepared by a data ethics consultancy with a number of recommendations, including for example the requirement that there should always be humans to have the last word, for privacy and data ethics to be built into any design by default, to fight data bias, to use AI in an inclusive way, and to develop standards on how to explain algorithms. [DK 15]

Think tank DataEthics

The think tank DataEthics was founded in 2015 by four women with backgrounds in law, journalism, and business. DataEthics.eu pushes the ethical questions of digital development including of Artificial Intelligence and automated decisions. [DK 16]

Rule-of-law think tank Justitia

The rule-of-law think tank Justitia has a general focus on rule of law questions, but is aware of digital and automated decision-making considerations and contributes to the public debate in the field. [DK 17]

Regulatory and self-regulatory Measures

Government commission on data ethics

In November 2018, a group set up by the Danish government—consisting of experts from business, academia and civil society—published a set of recommendations on data ethics. The report by the Data Ethics Commission [DK 18] agreed on nine recommendations including a permanent and specialised ethical council.

Political agreement on digital ready legislation

In January 2018, an agreement on digital ready legislation [DK 19] [DK 20] was aproved by all parties and the relevant guidance came into force in July 2018. [DK 21] This is set to be renegotiated in 2020 and replaces a previous similar agreement and guidance . [DK 22] The seven principles of the agreement concern 1) clear rules, 2) digital communication, 3) automated administration, 4) shared terminology and reuse of data, 5) safe and secure data handling, 6) use of public infrastructure and 7) prevention of abuse and mistakes.

The independent legal think tank Justitia raised concerns about a number of elements, including automated case management by public authorities. In particular, Justitia flagged the lack of rules in which cases would continue to determine when a human is needed to make a decision, and the lack of transparency in automated decision-making which would allow greater scrutiny. Justitia also raised concerns that digitalisation guidance did not sufficiently take into account the legal security and privacy of citizens. [DK23]

Implementation of the GDPR in Denmark

Denmark implemented the GDPR with legislation that came into force on May 23, 2018. [DK 24] However, a minority group of four centre-left parties in the Danish parliament were critical of the legislation when it was adopted. They said that the public authorities’ right to access and to combine the personal data of citizens was too intrusive, and that it meant that there was no obligation to inform citizens. [DK 25] [DK 26] The minority group protested against the “far reaching possibilities” of combining data from different authorities, for example “place of living, nationality, missed doctor’s appointments, unemployment, mental illness or drug abuse”. The minority group also emphasised the need to inform citizens about this compilation of data about individual citizens.

Algorithm transparency in automated decisions

Along with the implementation of the GDPR, Denmark takes the position that information about the ‘logic’ of automated decisions must be available to citizens: “When the data controller responsible solely has to inform about the ‘logic’ of the automated decisions, a more detailed description of the basis for the process cannot be demanded. The important thing must be that the affected person can understand the considerations underlying the process and how ‘the system’ reaches the various decisions”. [DK 27] Legal scholar Hanne Marie Motzfeldt fears that automated decision-making software could be used in public administration, while at the same time there are no control mechanisms in place such as technology that can trace patterns applied by the software.¹ In her analysis of the fulfilment of necessary ‘system transparency’ she refers to existing guidance by the Danish Ombudsman on the obligation of authorities to fulfil all necessary principles for public administration. These include the ‘officialprincip’—the implicit duty to investigate the facts of a case before making a decision. [DK 28] However, various cases lead her to the—explicitly tentative—conclusion, that administrative law fully secures transparency and control mechanisms.² Motzfeldt observes a lack of a “lively and qualified debate” about the legal aspects on the balance between technical possibilities and a wish for more efficient administration on the one hand, and the price to be paid by citizens’ trust, including demands for full transparency, on the other.³ Motzfeldt is the leader of the newly established Centre for Law and Digitalisation at the University of Århus. [DK 29]

ADM in Action

ADM in the Danish administration

Automated decisions are applied in Danish administration, often without much ado. For example, student stipends for higher education are decided by combining the student‘s online application with the information that he or she is accepted to undertake a course of education that qualifies for such a stipend, and the funds are then transferred to the bank account of the student. This process is based upon the law on study stipends [DK 30] and attracts little attention. But other fields of ADM do attract attention.

Banks and insurance

Profiling and automated decisions are present in the banking and insurance sector in Denmark. These activities are regulated via data protection laws and overseen by the Danish data protection authority, Data Tilsynet. In a more general comment on the dilemmas of the information economy, Rikke Frank Jørgensen, a specialised human rights expert, points at information imbalances, as well as the need to address the “fundamental discrepancies between a personalised information economy on one side and a society based upon respect for privacy and data protection”.⁴

Credit scoring

Following media coverage in 2005, Data Tilsynet produced a precedent decision for Experian, an international credit scoring company working in Denmark. The decision addressed the parameters for the credit prediction of individuals and companies using scoring systems called ‘Consumer Delphi (individuals)’ and ‘Commercial Delphi (companies)’. According to the company, the parameters used to make the decision included birth date, address and address changes, and open or closed registrations in a debtors’ register, including the size of the registered debt. In its decision, Data Tilsynet discussed the parameters included and described which parameters should be included and emphasised. In the case of an individual complaint, the decisive parameters for a given decision must be disclosed. [DK 31] Over the years, Data Tilsynet has repeatedly addressed company permissions, questions from Parliament, and complaints by consumers concerning their credit scoring practices. [DK 32]

Public discussion about the matter is not widespread, though specialised media focusing on digital and computer developments do pick up on the question. Media outlets have, for example, described how banks are capable of targeting customers based on their consumption patterns [DK 33], or by using interviews with companies offering profiles and predictions. [DK 34]

Car insurance

Car insurers offer rebates if drivers install a box [DK 35] to measure speed, acceleration, deceleration and g-force. The company offers a fixed 25% rebate for installing the box. Another company at some point pondered building a mobile app [DK 36] that included driving instructions and measurements leading to a quarterly, monthly, or potentially even more frequent adjustments to the car insurance premiums. This app, however, is currently unavailable.

Children in vulnerable circumstances – tracing model as part of the ‘ghetto plan’

An article in the Politiken national newspaper published at the beginning of 2018 caused a public uproar. Three local authorities had asked for exemption from the usual data protection rules to run an experiment [DK 37] [DK 38] to trace children with special needs from a very early stage. The model was named Gladsaxe, after the municipality in the suburbs of Copenhagen. The two other municipalities involved were Guldborgsund and Ikast-Brande, representing a rural, and a mixed rural-industrial community. The purpose was to trace children who were vulnerable due to social circumstances even before they showed actual symptoms of special needs. Based on previous use of statistics, the authorities decided to combine information about ‘risk indicators’.

The model used a points-based system, with parameters such as mental illness (3000 points), unemployment (500 points), missing a doctor’s appointment (1000 points) or dentist’s appointment (300 points). Divorce was also included in the risk estimation, which was then rolled out to all families with children. [DK 39] [DK 40] After the story about this system was published in Politiken—along with the government’s apparent plans to roll out the model all over Denmark—the public reacted strongly. The notion of a points-based system reached far and wide. Many refer to it—in jokes and irony—on a colloquial basis, such as “Oh no, I forgot the dentist. As a single parent I’d better watch out now…”. In addition, an evaluation scheme of children’s well-being and development at kindergarten was unveiled. Individual evaluations were prepared and stored without the knowledge of parents and in breach of existing legislation. [DK 41] While the latter is data gathering rather than automated flagging—and thus only creates material that can potentially be used for automated risk assessment—the public and political reactions to this scheme were strong, including the reaction from academia. [DK 42]

In spite of the public criticism [DK 43], the Danish government planned to roll out the early tracing model from Gladsaxe to the whole country. This is part of a larger ‘ghetto-plan’ to fight ‘parallel societies’. It is a plan that sets a number of criteria for a neighbourhood to qualify as a ‘ghetto’ and then introduces a series of special measures, such as higher punishments for crimes, forcing children into public day care at an early age, lifting the protection of tenants in order to privatise public housing, tearing down entire building blocks and—indeed—applying the automated risk assessment system for families with children. [DK 44] [DK 45] In September 2018 the minister responsible mentioned a planned legal act ⁵, but by December 2018 the speaker on legal affairs of the government coalition partner Liberal Alliance said to newspaper Politiken that the proposal had been shelved ^6.
Other publicly funded, automated risk assessment experiments in the field of social welfare are under development. For example, a project that measures chronically ill patients’ behaviour in order to estimate when or how further efforts are necessary. [DK 46] [DK 47] [DK 48] [DK 49] Significant government funding for investment in this field is allocated for 2018-2022. [DK 50]⁷ Data ethics consultants urge the general public to be mindful of democratic control, privacy, and ethical questions with such projects. [DK 51]

EFI – the failed tax collection system

EFI (short for one shared collection system, Et Fælles Inddrivelsessystem, 2005 - 2015) was initiated in 2005 to create a digital collection system for taxes at the local as well as at the national level. Before EFI, these taxes were collected separately and de-centrally. The new system had serious technical as well as legal flaws and led to the loss of billions of crowns for the public, due to expired or uncollected claims. It was halted in 2015. [DK 52] An official investigation found mistakes that could have led to the illegal collection of tax, wrong registration, or expiration of claims. [DK 53] According to legal expert Hanne Marie Motzfeldt, mistakes in EFI’s “data, design, programming and integration in the administrative bodies led to administration in conflict with the law”⁸. One of the problems was lack of insight into the processes: “Precise knowledge about the functioning of data and business processes that were ‘cast’ into the IT systems were largely placed with the IT provider” and not with the authority itself. Further “data and systems often were so badly documented that [the tax authority] did not have sufficient insights into them”. [DK 54]

Predictive policing

In the autumn of 2016, public tender documents and FOI requests obtained by journalists showed that the Danish police and the Danish police intelligence service had ordered a digital system from the US company Palantir. [DK 55] Tender documents showed that the system should be able to handle and make searchable very different data sources. These include document and case handling systems, investigation support systems, forensic and mobile forensic systems, as well as different types of acquiring systems such as open source acquisition, and information exchange between external police bodies. [DK 56] [DK 57] In that context, experts voiced criticism that this was a portent to making ‘predictive policing’ possible. [DK 58] The new digital system for Danish police and Danish police intelligence was adopted as part of anti-terrorism measures. [DK 59]

Two years previously, in 2014, an automatic license plate control system was introduced by Danish police. Using this system, police cars with a camera mounted at the front could automatically screen license plates, check them against several databases, and then indicate on a screen in the police car if there was a match alleging an offence. Human rights specialists have raised questions about the scale of surveillance. [DK 60]

Profiling and price adjustment

The Danish Consumers’ Council—a prominent, independent consumers’ association—explicitly warns the public of price discrimination. “Address, cookies and other personal information can be used to adjust individual prices on goods, so consumers do not pay the same price when shopping. This is unfair and makes the market opaque”. [DK 61] While not referring to individual cases, the group provides instructions to the public to avoid profiling via cookies. [DK 62]

Public sector data – Planning elderly care and HR document control

The Municipality of Copenhagen—responsible for very different tasks stretching from technical infrastructure and schools to social security and care of the elderly—cooperates with three universities in the capital region to use public sector data and develop automated procedures. [DK 63]

To improve the planning of care for the elderly, the municipality hoped to predict the needs of individuals. Data already logged about assistance, hospitalisation and from semi-structured text by caretakers were aggregated and combined to create an individual history. By analysing three months back in time, it was possible to predict with 80 percent precision when significantly more care would be needed, the municipality claimed. The logging and analysis did not change the need, but allowed for more targeted assistance and planning.

Another project concerned the human resources department, where the task was to control whether all relevant information about individual employees was obtained and correctly filed. This includes documents such as contracts, work permits and criminal records. The automated solution included a script to find the documents, place them in a cloud platform, read them with OCR translation, and use a self-made algorithm to find the relevant documents. The automated process was said to be 90 percent accurate due to bad scans of some documents, while manual checks were estimated to be 95 percent accurate. The cost of running the automated scan was estimated at 7,000 Danish Krone (just below €1,000), compared to an estimated manual workload of three months for ten people at 1,000,000 Danish Krone (or €134,000). [DK 64]

Udbetaling Danmark – automated payments and control of social funds

In Denmark, pensions, child allowances, unemployment support and many other social welfare payments are made by one centralised body called Udbetaling Danmark. This body has far-reaching access to data on individuals from a wide range of sources, which is regulated by the Law on Udbetaling Danmark. [DK 65] Data about a citizen from local municipalities, unemployment savings agencies and so forth are used to select a sample of cases for further control. In the first three quarters of 2017, this led to a selection of samples of just above two percent, a quarter of which was taken to further detailed control⁹. An analysis by Birgitte Arent Eiriksson, deputy director of the legal think tank Jusititia, relates the level of respect for privacy to the quality of decisions as estimated by a public control body, and reaches the conclusion, that “efficiency and surveillance” are rated higher than rights and the rule of law. Eiriksson‘s report asks for a deeper analysis including the origin and treatment of the data¹⁰, or in other words that transparency and proportionality need to be addressed.

IBM Watson & breast screening

In 2017, the Capital Region of Denmark entered into an agreement with IBM [DK 66] to test at least two AI projects per year using the company’s Watson system. Watson—marketed by name and with humanoid terminology such as a “new colleague who does not drink coffee” — was set to be used for routine preventive mammography screenings at two hospitals in the region. One of the arguments for using Watson was that there was a lack of qualified doctors who specialised in radiography. [DK 67] The new agreement was made in spite of reports about difficulties during a previous test with another tool, Watson Oncology, which according to media reports recommended life-threatening medication to cancer patients. [DK 68] While the evaluation of working with Watson Oncology was positive overall—and future interest was indicated by IBM, the health services and universities—the difficulties with the project were described as needing “further development and adaptation before the technology can be implemented for clinical use. For example, doctors and Watson only agreed on 27% of treatment suggestions. This was likely due, among other factors, to the fact that the system used had been trained in the US following American guidelines and practices”. [DK 69] In connection with these agreements, academics at the IT University of Copenhagen warned against being “deceived” by the new technology and called for better information of the public and of decision makers. [DK 70]

On a more general level, and endorsing the new technologies to further good health¹¹, professor of health and law, Mette Hartlev of Copenhagen University suggests that fundamentally new legislation is needed in the field of health and data to counteract discrimination, inequality, breaches of privacy, data security and so forth.

1 Motzfeldt, H. M. (2018). Retssikkerheden bør følge med den automatiserede forvaltning. I R.F. Jørgensen, & B. K. Olsen (red.), Eksponeret: Grænser for privatliv i en digital tid (S. 227-243). Gad, p. 238.

2 Motzfeldt, H. M., p. 240.

3 Motzfeldt, H. M., p. 242.

4 Jørgensen, R. F. (2018). Når informationsøkonomien bliver personlig (When Information Economy gets personal). In R. F. Jørgensen, & B. K. Olsen (red.), Eksponeret: Grænser for privatliv i en digital tid, Gad., p, 86

5 https://www.ft.dk/samling/20171/almdel/sou/spm/558/svar/1507910/1933577.pdf (amended after editorial deadline)

6 https://politiken.dk/indland/art6919255/Regeringen-har-lagt-sin-plan-om-overv%C3%A5gning-af-b%C3%B8rnefamilier-i-skuffen (amended after editorial deadline)

7 410 million Danish Crowns to be invested in the field from 2018 to 2022, press release by Danish government from October 2018 [DK 51]

8 Motzfeldt, H. M., p. 231.

9 Eiriksson, B. A. (2018). Social digital kontrol er på kant med borgernes ret til privatliv. I R. F. Jørgensen, & B. K. Olsen (red.), Eksponeret: Grænser for privatliv i en digital tid (S. 227-243). Gad., p. 34.

10 Eiriksson, B. A., p. 38 ff.

11 Hartlev, M. (2018). Sundhedsdata sætter patienters privatliv under pres. I R. F. Jørgensen, & B. K. Olsen (red.), Eksponeret: Grænser for privatliv i en digital tid (S. 227-243). Gad.