iBorderCtrl aimed at providing faster and more secure border controls through a complex toolkit of innovative technologies — including biometrics, emotion recognition, AI-based avatars, machine learning, and social media analysis.
The project’s Cordis page specifies that this meant “combining in one system state of the art technologies (from biometric verification to deception detection, document authentication and risk assessment).”
Consistently with the vision behind the expansion of large-scale IT databases (such as the reformed EURODAC, the newly created ETIAS and EES, and the EU Migration Pact), the system’s logic is one that includes a “pre-registration” phase for travelers — to which, however, the screening by virtual border agents, in the form of an avatar that would effectively constitute an AI-based lie detector based on facial microexpressions, has to be added.
This “comprehensive system” would allow “operators to use portable units at land border crossing points: road, walkway, train stations”, improving efficiency while saving the authorities time and money.
In an automation bias-prone fashion, adopting the iBorderCtrl system would also allegedly “increase scalability, reduce workload and subjective errors by human agents” and “increase the objective control with automated means that are non-invasive and non-time consuming.”
To put it more bluntly, “The main goal of this project is to promote biometrics as a mainstream method for traveler identification.”
According to the project’s Cordis page, “The iBorderCtrl research system relied on the integration of a multitude of technical innovations, employing both existing and proven technologies and novel ones or with lower maturity”.
The system was therefore developed in “modules”, which “used technologies at different stages of maturity and could be further developed or tested either individually or in combination”.
These included:
“•Pre-arrival registration of travelers previous to their arrival at the border: the checks take place in advance, allowing for much more computationally expensive methods to be deployed
• Assessment of attempts by the travelers to deceive border agents regarding the data indicated during the pre-arrival registration (purpose of their trip, length of stay etc)
• Personalized Avatar agent: the avatar agent that performs the interview can be personalized to gender and language (and, crucially, “ethnicity“, ed.) of the traveler
• Inclusion of new advanced biometric technologies: face matching and palm vein scanning
• Forecast of travelers flow at the border crossing points: statistics, trends and models help with better planning and resources allocation
• Analysis of historical data for border management improvement and identification of patterns of illicit border crossings
• Adaptive risk assessment of travelers: a multi-factor risk assessment, following a robust Multi-Criteria Decision Analysis methodology to support the decision-making of border guards.”
Some project deliverables obtained by AlgorithmWatch through access to information requests to the Research Executive Agency, namely D4.1 and D4.2, provide more granular details concerning the “iBorderCtrl Software Platform” fundamentally includes the RBAT (Risk-Based Assessment Tool), ELSI (External Legacy and Social Interfaces) and BCAT (Border Control Analytics Tool) modules.
Interestingly, and notwithstanding the abundant redactions in the project deliverables AlgorithmWatch managed to obtain, a technical description of the IT infrastructure of the iBorderCtrl system is almost fully in the clear in D4.1, including details on
1) The Traveller User Application (TUA), responsible of the “pre-screening procedure” (“Through this application, the travellers, following their consent, will be able to enter and update their personal information, upload travel related documents (such as VISA, passport) and travel information (tickets, hotel reservation, vehicles data etc.) and undertake the avatar interview. The application will send to the traveller a QR code upon completion of all required preregistration steps; when the QR code is handed over to the Border Guard at the BCP all information collected during the preregistration phase will be retrieved by the system in order to speed up the process.”)
2) The Border Guard User Application (BGUA), which “will be responsible to manage the border-crossing procedure.” (It “will be accessible and handled only by the Border Guards either through the portable unit or through a local terminal (i.e. PC) at the guards ’ booth at the BCPs. Through this application the Border Guards will be able to perform the Border Checks for each traveller crossing the BCPs: enter the travellers ’ and vehicles ’ documentation data during Border Checks, upload their travel related documents (such as VISA, passport, etc.), perform and upload scanners checks (fingerprints, palm vein, face matching etc.) and retrieve from the iBorderCtrl database the risk score per traveller.”)
3) The Border Manager User Application (BMUA), which “will be responsible to manage all the information collected and stored” (it “provides information concerning historical data as well as predictions of future days risk and traffic, providing statistics to assist the Border Managers on organizing and handling their everyday work.”).
Interesting details are also provided by D4.1 on
4) The External Legacy and Social Interfaces (ELSI) module, which was developed to “communicate with Twitter ’s API to recover data from consenting travellers for analyses”, which can then be integrated in “legacy databases” (SIS, VIS, and the forthcoming EES)
5) The Border Control Analytics Tool (BCAT), which “enables the combinatorial analyses of all data collected in iBorderCtrl utilizing statistical, machine learning and data mining approaches to discover new patterns and knowledge that can be used through the RBAT tool to enhance the performance of the system”.
iBorderCtrl was a famously controversial and yet influential project, which informed many others.
For example, project TRESSPASS “leverages the results and concepts implemented and tested (…) for land border control” in iBorderCtrl — and interestingly, “The TRESSPASS Consortium has been the Coordinator of all XP-DITE, iBorderCtrl and FLYSEC projects”.
The FLEXI-cross project also details “links with iBorderCtrl” in the Grant Agreement we managed to obtain, writing that “FLEXI-cross will employ experience and knowledge” developed over the course of iBorderCtrl, “such as the mobility concepts, processes designed to reduce cost/time spent per traveller at the crossing station”. “Technologies used” by iBorderCtrl “will be considered” as well.
Additionally, iBorderCtrl features among the “related projects” listed by — among others — the SMILE and FOLDOUT projects, and was “selected and studied” by the METICOS consortium for the developing of its own architecture (as illustrated in METICOS deliverable D4.4).
Crucially, this network of relationships extends well beyond research projects. Deliverables such as D8.3, D8.4 and D8.5, the periodic progress reports, make it abundantly clear that this technology is developed with a clear focus on “market uptake and exploitation” instead.
Writes D8.5: “The Consortium Partners have already included on their websites and dissemination channels their expansion of services based on the outcomes of the related research work project, to enhance both the dissemination of the project as well as to cultivate potential collaboration beyond the consortium partners on future projects that will build on top of the technologies developed in iBorderCtrl. In addition, iBorderCtrl already implements many of the features planned to be included in the Entry/ Exit (EES) and the European Travel Information and Authorisation (ETIAS) Systems both proposed by the European Commission to enhance the border control check procedures. Moreover, iBorderCtrl solution paves the way towards the interoperability of EU systems for security, border and migration management as envisaged by the EC. These are considered as major assets towards the implementation of a concrete exploitation plan.”
Piloting tests took place in Hungary, Latvia and Greece — but, more than five years after project completion — we still lack access to project deliverables detailing their operational scenarios and results.
What we know is scantily summarized in the project’s Cordis page: “The iBorderCtrl solution was designed and developed taking into consideration the requirements, guidance and expertise of the 3 end-users participating in the project, namely the Hungarian, Latvian and Greek Border Control officers. These experts also evaluated the iBorderCtrl complete system and individual components in multiple versions at the beginning and at the end of the research pilots, during extensive piloting tests that took place in 3 countries and in two pilot test phases. During the pilots, the Border Guards and Managers were familiarised and actively involved with iBorderCtrl and provided their constructive input.”
According to the “Reporting” section on Cordis, “The work was successfully completed and validated the basic concept; the pilot evaluation and usability testing provided some very interesting and promising results of individual modules and their performance.”
However, as noted by MEP Patrick Breyer, who took the system to Court, “Whether such “deception detection” technology works is highly controversial. The only “scientific” assessments of the technology have been published by Manchester Metropolitan University (MMU), which was part of the iBorderCtrl consortium. The MMU scientists have patented the technology and are selling it commercially through a company called Silent Talker Ltd. As the technology is based on machine learning, the developers themselves say they do not know what the system assumes are signs of deception.”
In fact, a much more cautios assessment is provided by researchers Kristina Suchotzki and Matthias Gamer in their 2024 paper ‘Detecting deception with artificial intelligence: promises and perils’: “Outside of books and movies”, they argued, “Pinocchio’s nose does not exist. There are no valid behavioral cues that differentiate robustly between liars and truth-tellers, and no physiological or neural signature has been identified that can unambiguously be attributed to deception. With the rapid advancement of AI, hope has been placed in its potential to improve currently underperforming lie detection methods. Unfortunately, these developments have mostly focused on technical aspects at the expense of a solid methodological and theoretical foundation.”
Actually, they write, iBorderCtrl is precisely an illustration of AI’s failures on providing a scientific basis to emotion recognition tech: “there are several ubiquitous problems in current research into AI-based deception detection, including the lack of explainability and transparency, the risk of bias, and a lack of theory” — a problem that “pertains to the underlying assumption that it is possible to identify a unique cue or combination of cues that is indicative of deception. Here, it is imperative to consider that decades of research have not been able to find these, nor have they produced a theory that can convincingly predict their existence”. In fact, “The case of iBorderCtrl illustrates these issues”.
More specifically to the project we’re analyzing, a box further writes:
“iBorderCtrl is a large-scale research and implementation project on border security that includes a module for deception detection, designed to infer malicious intent. This module is based on the system Silent Talker, an artificial neural network that uses a number of nonverbal behaviors to distinguish between truthful travelers and those who are lying about their identity, plans, or possessions. The Silent Talker system and algorithm are patented, and thus many details on its development and the algorithm itself are not publicly available. The few scientific publications on Silent Talker so far have described the development and test of the system in very small, non-diverse samples with fewer than 40 participants, of whom the majority were white European men and women (…) Here, the authors reported high correct classification rates. However, no further testing and cross-validation seems to have taken place in independent, larger samples before the project was funded by the European Union with 4.5 million Euros to be further tested and implemented in real life at European borders (one might wonder: why?, ndr). This reflects the high hopes that are put into AI-based security applications yet unfortunately also demonstrates that this often comes at the cost of basic scientific standards.”
And yet, the AI Act did not include migration among the banned applications of emotion recognition in the EU. Again, why? One might be tempted to answer: because systems such as iBorderCtrl — or TRESSPASS, whose MMCAT module was developed by the same consortium that worked on iBorderCtrl — must be deployable. After all, D4.1, p. 81, is explicit about the ultimate goal being real-life deployment: “iBorderCtrl is developed primarily by industry with the objective to one day move beyond research outcomes and onto real products on the market. To achieve that, focus was put on developing each aspect of the system in a way that allows for quick real world deployment following the completion of the funding period of iBorderCtrl.”
This fundamentally contradicts what the EU Commission, which manages the Research Executive Agency, has been saying publicly. Wrote a Reuters article in 2021 that according to emailed comments by the Commission “iBorderCtrl was not expected to deliver ready-made technologies or products. Not all research projects lead to the development of technologies with real-world applications”.
iBorderCtrl is arguably the EU-funded project in border surveillance that received the most attention from the media. It is hardly surprising, however, given the astounding amount of issues it embodied — and it still embodies.
Notwithstanding the obvious and severe ethical and even scientific issues raised by the project, its deliverables are still mostly shrouded in opacity more than five years after its completion.
Attempts at obtaining greater transparency failed in full or in part, including by MEP Patrick Breyer — and ourselves, as the deliverables which were (partly) shared suffered, and still suffer, from extreme redactions, in certain cases (such as for D1.2) so extensive as to render their release utterly pointless.
Breyer even brought the fight to the Court of Justice of the European Union, which however deemed that “commercial interests” outweigh public interest in this case. Breyer appealed, but lost (full decision). Nonetheless, he managed to obtain and publish some 45 (very partially released) documents, summarised here. One from 2017 seen by Wired Italia in 2021 described a covert lobbying effort — including through relationships with the media —to push for a change in the EU legal framework so that iBorderCtrl technologies, currently illegal, could become legal. The plan was blacked out, but Breyer managed to make it visible thanks to the help of an IT expert. The project caused huge opposition from civil society, leading to initiatives such as https://iborderctrl.no/.
We replicated Breyer’s request during the course of the Automation on the Move project, verbatim, and managed to obtain some additional (very partial) disclosures, namely of deliverables D8.1, D8.3, D8.4, D8.5, D8.7, D4.1, and D4.2.
Among them, D8.4 provides a clear example of how ethics assessments are consistently blacked out (e.g,, section 7, ‘Ethical issues monitoring during the reporting period’, only features a generic introduction in the clear, whereas the remaining pages and analyses are not readable).
This has to be added to the consistent — and in many cases complete — blacking out of details in documents that were already released to Breyer, and more specifically concerning
a) “risk of stigmatisation”, “profiling”, “automated decision making”, “risk mitigation measures”, “ethical risk awareness and monitoring”, “implementation of measures”, “transparency and training” and even “compliance with legal obligations” in D1.1;
b) “ethical risks and the different phases of iBorderCtrl”, “ethical concerns with regard to iBorderCtrl”, “Risk of Stigmatization and the Impact of Technology” — including “Definition and Classification of False Positives & False Negatives in the Legal Framework”, “Ethical implications of false positives”, and “Impact on the Individual vs. Impact on Other Travellers” —, “profiling of travellers”, “proposed risk indicators”, “proposed risk thresholds”, again “automated decision making”, “cultural and religious implications” in D1.2, which could serve as a textbook example of completely useless disclosure, given that the released document is almost fully blacked out.
The project also dangerously tries to normalize the adoption of emotion recognition technologies (“The IBORDERCTRL system has been set up so that travellers will use an online application to upload pictures of their passport, visa and proof of funds, then use a webcam to answer questions from a computer-animated border guard, personalised to the traveller’s gender, ethnicity and language. The unique approach to ‘deception detection’ analyses the micro-gestures of travellers to figure out if the interviewee is lying”) that are still mostly considered unscientific by academics and in the literature (cfr. above).
Additionally, a problematic idea of automated “risk assessment” is described in a peer-reviewed publication, according to which it should be performed in this quasi-fully automated fashion: “the system will interact with the traveller autonomously by deciding on which questions to ask, how to behave (i.e. the avatar can adapt its behaviour to the behaviour of the traveller, such as acting rather sceptical if an answer seems to be not correct) and finally to assess the overall risk stemming from the traveller based on the information provided in the pre-registration phase and the results of the deception detection. This risk assessment will be delivered to the border guards, assisting them in their decisions during the actual border check in stage 2 on whether a person is allowed to cross the border, or if a thorough check might be required.”
Lastly, and worryingly, ideas developed in iBorderCtrl led to the development of further controversial applications in EU-funded projects. One is the MMCAT module of the TRESSPASS project, described in TRESSPASS D6.4 as “an interview system which extracts indicators from the video recording of interviews such as micro movements pertaining to facial expression, gesture and posture (smiling, blinking, hand movements, leaning etc.). The hypothesis is that the performance of such movements, or the frequency/intensity with which they take place can provide indicators of insincerity or nervousness and thus influence the level of risk to be assigned to the interviewed.”
‘Do Europe’s borders need multi-faceted biometric protection?’ even argues that the iBorderCtrl project “could open a path to the next generation enhancement of the EU’s integrated border management system”, turning it into a role-model for EU-funded projects to come.
When it come to respecting fundamental rights, the project writes: “The system was designed also to not only respect but improve transparency for travelers, respect their fundamental rights, ownership and control of their data, and quality and enjoyability of their border crossing experience.”
Solutionism is also in full display in the same publication, as project participants write that “Without such innovative technology, a traveller could easily face the same queues and waiting times as in the 1980s and 1990s, resulting in the benefits of a dynamic cross-border flow being lost. This is the core advantage of the Schengen regime, and slowing the border crossing process could result in a direct cut in the average 2.3% GDP increase created by Schengen”.
The project is also clearly born with the needs of border officials and operators in mind, rather than those of migrants. Note for example how the project portrays itself as a “Smart lie-detection system to tighten EU’s busy borders”, which is necessary to fight crime and terrorism at EU borders: “an ‘intelligent control system’ facilitating – making faster – border procedures for bona fide and law-abiding travellers. In this sense, the project is aiming to deliver more efficient and secure land border crossings to facilitate the work of border guards in spotting illegal immigrants, and so contribute to the prevention of crime and terrorism”.
This buys into the more general discriminatory assumption embodied in further projects, such as NESTOR, according to which automation must be used to discriminate among those who deserve trust and therefore minimal checks, and individuals who are instead deemed high risk and therefore potential criminals and terrorists to be thoroughly checked. All of this should mostly be automated: “This pre-screening step is the first of two stages. Before arrival at the border, it also informs travellers of their rights and travel procedures, as well as providing advice and alerts to discourage illegal activity. The second stage takes place at the actual border. Travellers who have been flagged as low risk during the pre-screening stage will go through a short re-evaluation of their information for entry, while higher-risk passengers will undergo a more detailed check. Border officials will use a hand-held device to automatically cross-check information, comparing the facial images captured during the pre-screening stage to passports and photos taken on previous border crossings. After the traveller’s documents have been reassessed, and fingerprinting, palm vein scanning and face matching have been carried out, the potential risk posed by the traveller will be recalculated. Only then does a border guard take over from the automated system.” It goes without saying that such a system is ripe for algorithmic abuse.
