SMILE aimed at creating a mobility concept with biometrics as components.
According to deliverable D2.6, the project designed, implemented, and evaluated “several novel prototype architectures, for the accurate verification, automated control, monitoring and optimization of border traffic (pedestrians and vehicles) at Land Border Infrastructures (BCPs and Border Gates).”
The project introduced ideas that have long been and still are influential among EU-funded projects in border security, such as
1) using “mobile equipment for biometric verification” for border authorities,
2) obtaining “efficient and unobtrusive, on-the-move security control for individuals and groups” (D7.6), and
3) the notion of a “preregistration” phase — described as “a similar concept to the check-in process used in air travels, whereby the travellers use the SMILE application prior to their journey to submit information regarding their trip.”
Another popular idea, both in EU-funded research and EU migration policy, is that this preregistration informs a risk analysis whose results — which are “not communicated to the traveller/visitor” — “will dictate if the associated traveller/visitor will be routed to a fast lane/green line or to a standard lane/red line respectively” (D2.6).
This technological toolkit would allegedly result in both enhanced security and reduced queuing time, as the SMILE application “ensures that border authorities have all the necessary information in advance, enabling them to run any checks off-line.” This way, everything would work smoothly: “By the time the travellers arrive at the border, they only need to make one final identity verification before quickly completing their crossing.”
Another reoccurring idea is to let automation determine who is considered low and high risk traveler. As deliverable D7.6 puts it: “High-risk travellers are led in a different line for further verifications when they arrive at the BCP, while at the high security building, high-risk attendees are informed in advance that they cannot enter at the event and if they try to attend, the entrance would be forbidden.”
The SMILE page on Cordis claims that the project “aims to evolve beyond the state of art in various areas.”
It introduces the notion of “pre-registration” for land border crossings, to be performed through a SMILE application to “pre-input” all the necessary information (including biometrics).
It also developed “multifactor”/“multimodal” biometrics, to enhance “the recognition performance and avoid cases where a biometric could fail (e.g. the traveller’s finger is injured thus failing the fingerprint recognition). By relying on multiple biometric traits, the false rejection and the false acceptance rates will be significantly lowered.”
Controversially, “Soft-biometrics” were to be adopted to enhance “the performance of hard biometric modules as well as to assist in group identification.” According to SMILE’s Cordis page, this would include “facial hair (moustache, beard), eyes colour, skin colour (ethnicity), gender, age group (child, adult, senior) and eyewear.”
Security features were also being developed, including a technique to “provide un-replicable encryption keys between the various SMILE devices,” and better safeguard data integrity and privacy.
“Key technologies” involved in the SMILE project are, in summary: “Homomorphic encryption mechanisms; Physical Unclonable Function (PUF)-based security solution based on photonic technologies; Cloud Trust Protocol (CTP) a mechanism that generates an evidence-based confidence for cloud clients; Soft biometrics; Single Search GUI, a web-based platform for travellers, land border staff & authorities; An embedded security element (chip) ensuring that data is stored & exchanged only to authorized applications & people; Match-On-Card Face & Voice; Autonomous Mobile Device for Border Control.”
The “Related projects” page includes: TRESSPASS, PROTECT (“PROTECT’s primary goal is to build an advanced biometric-based person identification system that works robustly across a range of border crossing types and that has strong user-centric features.”), iBorderCtrl, BODEGA (“BODEGA’s aim is to develop future border checks with human factors expertise in order to enhance efficiency, border security and traveller satisfaction. The project focuses on the changes to traveller processes and border controller work due to the introduction of smarter border control systems like automated gates and self-service systems based on biometrics.”), ABC4EU (“ABC4EU stands for Automated Border Control Gates for Europe. It is an EU wide project and involves a Consortium of 14 partners from 7 different countries. The aim is to make border control more flexible by enhancing the workflow and harmonizing the functionalities of Automated Border Control (ABC) gates, which are only one example of automation.”), C-BORD (“C-BORD develops five technology pillars to enable next-generation container NII at EU sea and land borders.”), ARIES (“ARIES aims to set up a reliable identity ecosystem combining mature technologies for a high level of assurance, such as biometrics or use of secure elements, with innovative credential derivation mechanisms. ARIES advocates management practices that ensure user-centricity and flexibility when it comes to secure and privacy-respecting virtual identity management processes. The project will help to tangibly achieve a reduction in levels of identity fraud, theft, and associated crimes.”), and VICTORIA (“VICTORIA is a 3-year EU-funded project launched on 1st May 2017 which will deliver an ethical and legally compliant video analysis platform that will accelerate the video analysis tasks of Law Enforcement Agencies.”).
In regard to TRESSPASS, an interesting fact is highlighted in Newsletter #3 on a SMILE-TRESSPASS collaboration: “On June 26, 2019, SMILE representatives attended the TRESSPASS H2020 Stakeholder Advisory Board which acts as a high-level external body of experts, composed of research, industry and end-users representatives. In the context of the two projects collaboration, SMILE representatives had the opportunity to present the project’s concept while approaching a specialist audience in the border security domain. This raises some questions. Is it common for members of an EU-funded project to be included among the stakeholders for another EU-funded project? And is it appropriate to include them in the activities of a high-level “external” body of experts for another EU-funded project?
Full integration with EU large-scale IT databases is also aspired: While describing the system’s ideal operations, deliverable D2.6 notes that “SMILE interacts with the external Databases (e.g. VIS, SIS II) for finding out if any alert has been issued for the preregistered travellers and/or their travelling vehicles (if any), or if they have a valid VISA (for TCN cases).”
The same deliverable specifies that SMILE can retrieve “input from scanning devices utilised currently in the legacy BCP systems such as ANPR, devices counting the number of passengers in a vehicle and so on.”
There were three pilot demonstrations: two border control posts at the Romanian-Hungarian and Bulgarian-Romanian borders (in Nădlac and Ruse, respectively), plus a third “supplementary” scenario. This last pilot in Nădlac and Bucharest was to demonstrate “SMILE’s capabilities to be utilized in an area outside border controls, thus expanding the project’s exploitation potential” —showcasing an application “aimed towards a high-security building security.”
A detailed assessment of the pilots is provided in deliverable D7.6: “The border crossing for land infrastructures scenario constitutes the core scenario of SMILE and its main reason of existence. Therefore, two pilot sites (…) were dedicated to its evaluation. The two pilots applied the same use case scenarios with only minor differences, such as the automated license plate recognition that took place only in Ruse. The obtained scores for both pilot’s,” argues the deliverable, illustrated “the successful outcome of their evaluation.”
The third, “supplementary” scenario — the one “that shows the potential of SMILE for application to other areas”, and namely access control for high-security buildings — was also tested. But “In this case, the scores from the batches of tests were moderate.” The authors blame the COVID-19 pandemic, which “had a high impact.”
Consequently, due to “the importance of the other two pilots, a priority was given to them to resolve any occurring issues, keeping always in mind that there is sufficient time to resolve the issues of the SPP pilot (the third one, ed.) and have the appropriate tests. Nevertheless, pilots had to be suspended due to the pandemic, depriving valuable time that could be devoted to a new series of tests with updated versions of the SMILE system. Moreover, contrary to the other pilot sites, in this case there was no remote access to the pilot, due to security reasons. This restriction hindered the efforts of the technical partners who could not interact freely with the installed system.”
All details of the three pilots’ various iterations are included in deliverable D7.6. We learn that, in the first pilot, “face recognition and Soft-biometrics modules worked very well except some cases due to the fact that they are ambient light dependent. Sometimes travellers had to get out of their vehicles in order to have successful detection. Other modules worked flawlessly and exhibited better performances regarding both time requirements and accuracy.”
Even in its third iteration, the first pilot didn’t get satisfying enough results from face recognition: “Face recognition does not work steadily with low ambient light, slowing down additionally border check”, the deliverable notes. “It takes more than a minute to capture face biometrics with low or poor lighting and it exceeds desired KPI value. Moreover, sometimes BCP officer has to ask the travellers to get on a brighter place in order to get a successful detection.”
In the second pilot, “Iris and fingerprint modalities” were not piloted only because of the COVID-19 outbreak.
As for the third pilot, the deliverable again refers to the circumstances: “Unfortunately, in this pilot, the COVID-19 pandemic had a high impact.”
However, as the “border crossing for land infrastructures scenario constitutes the core scenario of SMILE and its main reason of existence,” D7.6 concludes that the “obtained scores for both pilot’s range between 0.88-0.95 illustrating the successful outcome of their evaluation.”
The SMILE project is clearly born out of efficiency needs, and more generally to reduce the time and workload of border authorities. As for the benefits to travelers, “convenience” — rather than protection of their fundamental rights — is advertised right from the start. And even though “improving life quality of EU and non-EU people crossing EU borders” is allegedly part of the project, 9 out of the 10 mentioned goals concern LEAs and security instead.
Deliverable D7.6 puts it very clearly: “The main contribution brought by SMILE for border authorities and SPP (Serviciul de Protecție și Pază, the Romanian Protection and Guard Service, in charge of protecting national security, ed.) is the reduction of queues at the BCPs/High security building, by reducing the waiting time for travellers/attendees.”
This is even more noticeable given SMILE’s interest in popularizing mobile biometric verification for border crossings, and its introduction of the use of biometric information for pre-screening procedures.
“Results in Brief” on Cordis are telling: thanks to mobile biometric tools, checks will be “faster and more convenient,” all the while collecting a wide range of traveler data that, according to project coordinator Dr. Dimitrios Tzovaras, include “date and time of crossing, personal information and biometrics, as well as the means of transportation (car, bus, etc.) and anyone accompanying them, such as family members, friends and coworkers.”
More generally, one of the main objectives is the “introduction of a biometric based pre-registration service as a key element in land border crossing points of the future, so as to minimize the risk level by changing the level of BCP control from standard controls to strong controls”. This is fundamentally tied to a “risk analysis”, explains Newsletter #1, the analysis being essentially based on data obtained from “national and international databases.”
In fact, in Newsletter #2 a link between the EU’s interoperability proposal for information databases (SISII, VIS, EURODAC, ETIAS, ECRIS-TCN and SLTD) and SMILE is established, arguing that “SMILE will facilitate the proposed framework, as it will integrate with all the systems involved in the proposal and address all the above-mentioned elements.”
Interestingly, the project’s website includes an “Imagine” section in which the ideology behind SMILE is striking. The readers are invited to imagine a “border crossing point that could provide an integrated solution, friendly both for travellers & border officers, based on the usage of biometrics that we already use in our everyday life through our smartphones. (…) A solution that leverages a blend of the best existing technologies & innovations in the fields of encryption & cloud computing, which will together offer an unobstructed flow of people, robust border checks and enhanced data privacy & security.”
Data privacy seems to be the main concern in the ethics screenings performed for the SMILE project. For example, WP8 concerns “Legal and Ethical assessment on data privacy, Adaptive Ethics, Standardization and Regulatory Activities” but the ethics assessment — as shown, among others, by D8.9 (concerning the “legal and policy requirements”) and D8.12 (on data privacy and ethics for the project pilots) — seems to be mostly dealing with data privacy and legal requirements, and not with the profound societal implications of the tech being developed, even though these tech solutions were to be socially accepted.
A “privacy and ethics impact assessment” was conducted, and a “How to do it right” framework was developed for the assessment of ethical and societal impact of biometrics tech developed in SMILE (cf. a 2019 presentation and the “How to Do It Right: A Framework for Biometrics Supported Border Control” paper by Mohamed Abomhara et al. published for the International Conference on e-Democracy).
Among the project’s publications we can also find a paper with the telling title “Border Control and Use of Biometrics: Reasons Why Right to Privacy cannot be Absolute.”
This paper and D8.6 reflect on the moral justification of biometrics in border controls, which is — according to the paper — “essentially utilitarian.” This requires a warning — “serious remarks” — “over the use of biometrics technology for large populations, especially if the consequences lead to social exclusion, either as a result of the individual being unable to reliably enroll or verify their data, or simply not having confidence in the system and avoiding having to interact with it (for example veiled Muslim women, ed.).”
In this context, the paper argues, utilitarianism fails. “Certainly, when it comes to border control and the use of biometrics technology to increase border security, monitor migration and combat identity theft and fraud etc., the argument is essentially utilitarian (utilitarianism theory) where the collective right of a group (group interest) is balanced against the rights of the individual. It makes the individual simply a means to the ends of the majority. However, this is the wrong argument. Wickins (…) and Townend (…) argue that public interest must be judged by considering the balance between individuals, i.e. the rights of single individuals must be balanced against other single individuals if individuals are not to be used instrumentally.”
And despite this fundamental rebuke of the very logic behind the project (can unobstructed, seamless and mostly automated border crossings actually respect the rights of each and every traveler, rather than maximize the efficiency of cumulative border crossings?), the paper claims that “individual acceptance of biometrics technology should be actively promoted through ensuring transparency of decision-making, clear policy regarding the purpose of biometric technology and how it is used, as well as increased measures dedicated to preserve personal rights and personal data protection. Since greater use of personal data impacts upon human rights, there needs to be an honest and assertive study of what the risks are to personal rights and privacy as well as how these risks are mitigated.”
Two deliverables in WP8 (D8.7 and D8.12) detail the work of the “Ethical helpdesk board.” “An Ethical helpdesk board consisting of five experts with extensive experience in privacy and ethical issues independently monitor and review the activities and outputs of the project.” D8.12 specifies that this features “an external member/advisor.” Reflections by these experts mainly concern informed consent, privacy protection, and confidentiality.
An important disconnect between the desiderata of border guards and travelers concerning biometric checks is identified in a project paper: “As a result of our investigation, two conflicting trends emerged. On one hand, border guards argued that biometric technologies had the potential to be a very effective tool that would enhance security levels and make traveler identification and authentication procedures easy, fast and convenient. On the other hand, travelers were more concerned about the technologies representing a threat to fundamental rights, personal privacy and data protection.”
