D4FLY aims to obtain seamless and secure border crossings through novel biometric technologies.
Reads the project description: “The overall D4FLY concept is to identify, reduce and avoid vulnerabilities in the identity lifecycle of modern border management, thereby strengthening European Smart Borders against serious and organized crime. It does this by exploring, developing and validating new technologies for identity and document verification – including biometrics-on-the-fly, and even explores how this could work in risk-based border management concepts” (such as the ones embedded in the EU Migration Pact, ed.).
Furthermore, and consistently with other EU-funded projects, D4FLY promises to “augment the current capabilities and capacities of border authorities in countering emerging threats in document and identity verification (e.g., forged documents, impostor fraud, morphed faces) at manual and highly automated border control points and in the issuance process of genuine documents.”
Concretely, “The D4FLY solution will consist of a border control kiosk geared with enhanced enrolment, verification, and detection capabilities; smartphone applications for improved performance and verification capabilities, and a non-stop on-the-move system for biometric verification. (…) Novel sensor hardware based on advanced lightfield cameras and novel algorithms developed in the project will enhance verification accuracy and robustness via the combined usage of 2D+thermal face, 3D face, iris, and somatotype biometrics. Additionally, analytical means to identify known criminals based on somatotype and 3D face data generated from mugshots and observation data will be developed.”
To avoid ambiguities, the Consortium specifies that “With “real-on-the move” we refer to the term “real-time” (i.e. fast) identity verification of travelers. The goal of the project is to be able to provide solutions that can verify the identity of a traveler as he/she passes through a specifically designed area, monitored by cameras, while movement flow remains uninterrupted.”
D4FLY featured several technological innovations. Among them:
1) Biometric technologies for the verification of traveler’s identities on-the-move through 3D face, iris, and somatotype recognition;
2) Smartphone applications for enhanced traveler verification;
3) Thermal and multispectral imaging for counter spoofing;
4) Advanced morphed face detection algorithms through Convolutional Neural Networks, Deep Neural Networks, and advanced methods for impostor and document fraud detection;
5) Computer vision algorithms for automated passport forgery detection.
“Related projects” are explicitly listed on the project’s website: METICOS, TRESSPASS, ANDROMEDA, SMILE, PERSONA, PROTECT.
In terms of risk assessment, deliverable D6.5 writes: “There are several EU projects that have developed technologies related to border risk analysis. Two representative examples which relate closely to D4FLY” are projects TRESSPASS and FastPass.
Deliverable D4.3 adds that scenarios to test prototypes and demonstrators were developed “Building on the experience of previous EU-funded projects on similar topics (like e.g. [PROTECT])”.
D4FLY was deemed “of interest” by Frontex in 2020, D4FLY is also part of the BES Cluster of EU-funded projects in Border External Security, lead by METICOS.
According to deliverable D4.8, D4FLY technologies have been “integrated, tested and demonstrated” in four scenarios: 1) for enhanced document verification, focusing on both travel and breeder documents, during three demonstrations that took place in the Netherlands between October 2020 and April 2022;
2) for “Highly automated border post with travellers arriving in waves like typically occurring in a train station or in a cruise ship terminal” at the Piraeus Port in Greece (February 2022);
3) for “A border post at a land border with manual passport controls without automated biometric verification systems” during a field test in Lithuania (August 2021); and
4) a field test and the final demonstration (June 2022) in Portsmouth, UK, for “A coach scenario where border guards verify documents and identities without a fixed post in a crowded confined space”.
Interestingly, among the conclusions of the deliverable we can read that “The initial versions of the prototypes have been used in the first field tests. Those versions have been improved based on feedback mainly from end users participating in the field tests.” And even though “Some of the integration improvements have been challenging”, test results were considered to be generally positive.
Concerning the outputs of D4FLY, a December 2023 Press Release by the project coordinator, the German “identity solutions” provider Veridos, wrote:
“With a total of 19 different partners from 11 different countries, including institutes, universities, border control authorities, and companies from industry and business, Veridos, as the overall coordinator, developed products and technologies between 2019 and 2022 that, depending on their degree of maturity, are either already in use today, are close to final product maturity, or require further basic research that will continue even after the official end of the project.”
2019-2022 is precisely the lifespan of the project, which is said to be ushering in “a new era of border control technology”.
Ethics and privacy are explicitly tackled as actual issues throughout the project, writing for example that “The D4FLY technological tools aim to verify travellers’ identities through the use of biometrics and detect fraudulent breeder and travel documents through the use of artificial intelligence”, and therefore “their development and deployment has the potential to impact fundamental rights and EU societal values”.
On the bright side, “D4FLY includes a dedicated Work Package to understand this potential impact and to mitigate and avoid any negative consequences, harm, or infringement of rights. Simultaneously, the consortium seeks opportunities to promote EU values and facilitate the exercise of fundamental rights.”. Also, “Special attention” is paid to, among other factors, “Bias and discrimination in biometric verification tools and in datasets”, “Automation Bias” and “Societal Acceptability of the D4FLY tools”.
Specific safeguards were put in place: “To avoid risk to these values and rights and to protect and promote them, the consortium has completed the following assessments and reports: 16 ethics requirements deliverables focusing on data protection and research ethics; Data Management Plan to track the flows and protection of data in the project; 4 Data Protection Impact Assessments for the collection of biometric data to ensure the protection of the rights of participants; Privacy Impact Assessment (PIA) to identify and mitigate any potential risks to individual and group privacy and to identify privacy enhancing technologies to develop; Ethical Impact Assessment (EIA) to identify and mitigate any potential risks to EU societal values or fundamental rights and to identify steps to promote values and the exercise of rights. Ethics monitoring of all facets of the project is ongoing and led by partner Trilateral Research”.
On the other hand, one might wonder whether concerns on ethics and privacy were actually operationalized. Deliverables are mostly very high level (and vague): e.g., D3.1 (containing very broad definitions and principles in ethics); or the 2020 workshop on privacy in Amsterdam, again to vague results.
Also problematic is the “risk assessment” function detailed in deliverable D6.5 (for the technical aspects) and D3.3 (for the ethical ones): “It is relevant from an ethical perspective”, wrote the latter, “that the D4FLY project contains a task in which various kinds of metadata will be fused together in order to provide an automated risk assessment of a particular traveller. The metadata to be included are: the verification score generated from the verification between the kiosk enrolment and the biometric corridor, incidents of face morphing and presentation attacks, travel pattern anomalies and incidents of document fraud. End users have expressed interests in pulling data from Passenger Name Record Data and the Entry Exit System as well. The risk assessment will appear as a traffic light system for border guards by which travellers are given a red, yellow or green light corresponding to high, medium and low risk.” Were mitigation strategies adopted to avoid abuse and bias through such system?
Considerations concerning the widespread adoption of biometrics for identification of people on the move are not promising, in this respect: “The ethical concern is that something akin to Cartesian mind/body dualism is implicitly adopted by border guards or technology designers whereby ethical values such as dignity or autonomy are considered to be anchored in mental or psychological features alone and the human body becomes objectified and seen only as a data source. However, bodily integrity and respect for another’s body must be maintained. Although this report does not offer a specific mitigation for this ethical concern because it falls outside the scope of the D4FLY project, the question of how the use of biometrics generally might affect an individual’s or society’s view of the human body is worth considering by any project partners designing or using biometric tools”.
Section 4 of D3.3, “Ethical concerns”, does detail several mitigation strategies for ethical issues — but again, mostly weak, vague and scarcely applicable. What is more, the main objective of the whole work package is contentious. Writes D6.5: “The main objective of this task is to develop and test a universal system to automatically build a risk profile and determine risks related to identifying travellers by gathering and analysing information from a variety of sources, which includes document and travel metadata, as well as metadata derived from fused biometrics (…), alternative technologies for identifying people (…), tactical and travel pattern anomalies (…) and interoperable databases and other data sources available to the border authorities. Such a risk analysis system would be beneficial for enhancing the border security and speeding up the border check process. The output from the risk assessment based on metadata fusion could be presented in a universal warning system to show a flag to the border guards to simply indicate the risk level and help them quickly identify any suspicious patterns.” This system is ripe for abuse.
Lastly, it has to be noted that the general tone throughout project documents is informed but at times dismissive, such as in this passage: “it is critical to note that the D4FLY project is not designing tools for ubiquitous surveillance through facial recognition. All of the project’s tools are meant to be implemented at border crossing points where travelers have already consented to having their identity authenticated by border authorities. As such, the tools under development will not add additional sites of surveillance to society. Rather, they seek to make current identify verification practices more accurate and efficient.”
