TRESSPASS is (yet another) technological solution to “effectively combat border-related crimes,” integrated with “other risk-based border management systems.”
In a typical solutionist fashion, TRESSPASS defines the “advanced technologies for border monitoring” that it aims to develop as indispensable, “in light of recent crises across Europe and worldwide, along with numerous border conflicts and illicit activities.”
The TRESSPASS solution is designed for “air, maritime and land (including car and train) border crossing points, and specifically also travel routes that combine different modalities.” It does not apply to “border crossings outside of border crossing points, such as happens with boats of refugees on the Mediterranean.” Smuggling and irregular migration are however listed among the top “threats” that the project aims to tackle more efficiently.
According to an analytical description on its Cordis page, TRESSPASS will
“(1) develop a single cohesive risk-based border management concept
(2) develop three pivoting pilot demonstrators
(3) demonstrate the validity of the single cohesive risk-based border management concept by using red teaming and simulations
(4) prepare for the further development of this concept beyond this project by linking to other known risk-based border management projects (in- and outside EU, within EU research frameworks and on national levels), and describe how their results contribute to a single cohesive risk-based border management concept.”
Several technological tools and components have been developed within TRESSPASS.
Deliverable D6.4 illustrates the “TRESSPASS RBBM (Risk-based border management) system,” which “is comprised of a risk management framework (TRAM, TRESSPASS Risk Assessment Method, ed.), a catalogue or index of risk indicators (RIs), technological system architecture (technological components) which gather, transform, and redistribute the risk information about the passenger and, of which, is used to fuel the calculation and attribution of the passenger’s risk score and a guiding legal and ethical framework and a Concept of Operations (CONOPs) guide.”
In terms of components, deliverable D9.4 lists VTC (Video Tracking Component), TCSS (Counter Spoofing Component), RTBA (Real Time Behavioral Analytics), MMCAT (Multi Modal Communication Analysis Tool), and WI (Web Intelligence).
D9.4 also provides further details:
1) “VTC uses the video recordings of walking people to extract their moving trajectories. These trajectories are then processed by the RTBA component in order to identify ‘deviant’ or ‘ abnormal’ walking patterns”. It is explicitly assumed that “following an ‘abnormal’ movement pattern can be considered a risk indicator for border checks.”
Initially, the datasets for development relied on footage of actual people, who did not give their consent: “For the development and testing of the VTC components, VV provided a list of 6 datasets, collected between 2009 and 2016 in different countries in and outside of the EU, including China and the US, and containing the images of 72 to over 9.000 persons per dataset. The datasets consist mostly of full-body images of people while they are walking, mostly in public spaces. According to VV, no consent was asked to participants for capturing the videos.”
Even worse, “due to the need for some clarifications, the DPIA for the VTC was produced by VV too late to be submitted for the Ethics Check in October 2019. Therefore, there was no opportunity for the ethics reviewers to assess the use of these datasets, nor for the Consortium to ask for a feedback by the Ethics Reviewer.”
A harsh assessment ensued: “Considering the lack of documentation in the ethics material provided for the Ethics Check, the lack of consent by participants during collection of the mentioned pre-existing datasets, the places of collection of data and also the fact that the datasets collected in Europe were collected before entry into force of the GDPR, the WP9 leader after consultation with the EAB and the project/technical coordination, considered the ethical implications of the use of such datasets to be considerable and suggested to seek for alternatives to using these datasets, i.e. by using synthetic data.”
Eventually a new GDPR-compliant dataset was created during a “recording day” at Lelystad Airport (NED).
2) “TCSS uses thermal infrared sensors in order to detect whether a person is wearing a mask.” Wearing a mask at a border crossing point is considered in the TRESSPASS risk-based framework a spoofing attempt which influences risk indicators.
3) “MMCAT is an interview system which extracts indicators from the video recording of interviews such as micro movements pertaining to facial expression, gesture and posture (smiling, blinking, hand movements, leaning etc.). The (extremely contentious, ed.) hypothesis is that the performance of such movements, or the frequency/intensity with which they take place can provide indicators of insincerity or nervousness and thus influence the level of risk to be assigned to the interviewed.”
Issues with the adopted datasets arised in this case as well: “TNO and VV provided information pertaining to a dataset used for the development of the tool. The dataset contains videos of criminal trials in the US, on which witnesses’ and defendants’ faces are clearly visible. Moreover, the datasets are not anonymised. At the time in which the discussion on this topic took place, however, the use of the dataset was already completed.”
Would pseudonmization work? The deliverable notes, “names are kept separately but (biometrical) re-identification is possible.”
The occurrence of personal data issues is documented elsewhere: “A question that was unanswered after the Ethics Check was whether during the Dutch Pilot data collected during a previous project, Seamless Flow, would be used,” notes the deliverable. “This would have had significant ethical implication, given the unclarity about consent provided by participants in Seamless Flow for further use of their data, the sensitivity of data involved (biometric data) and the potential transfer of data from outside the EU (given the participation of an Airway Company from Hong-Kong in Seamless Flow). The matter was discussed in the EAB and with the involved CMs and the decision was taken not to use any data from Seamless Flow.”
The conclusion from deliverable D9.4 is stark: “The TRESSPASS project, which aims at introducing a new concept for border-crossing checks, is a very challenging project from an ethics point of view. The effort of T9.1 in the second project period focussed on consolidating awareness for ethical issues among the consortium members. Compared to the beginning of the project, there has been considerable progress in this respect, so that taking into consideration ethical aspects while conducting research, development and tests has become common sense (note the phrasing “has become,” ed.) among the CMs. Of critical importance towards this achievement has been the competent and committed support and advice provided by the two external members of the EAB, as well as the high priority that has been given to ethical consideration by the Project Coordinator and Technical Manager.”
Deliverable D8.5 adds details on additional components:
4) “DRAS provided real-time risk assessment for each traveler based on continuously receiving traveler risk indicator data from Data Fusion Analytics (DFA) module, calculating traveler risk score per threat and overall risk score, using weight-based formulas, and providing traveler risk profile to the border guards through the Command-and-Control front-end user interface.”
5) With C2, a “centralized component,” or “central database” gathering all “risk information”, border control operators “would have more information available to them to know who to direct to further checks and who to not. Having the risk information located in one centralized component was already considered beneficial and if the risk information was more detailed it would decrease the difficulty of the risk assessment task for the border guards,” adds the deliverable, “subsequently reducing their workload and pressure, whilst also providing more secure risk information.”
Book chapters and publications largely address deep learning techniques applied to the “detection of 3D face masks with thermal infrared imaging,” “anomaly detection with noisy and missing data,” “automated real-time risk assessment for airport passengers,” and even a “scoring algorithm for abnormal traveller behaviour in border crossing areas”.
TRESSPASS is part of the BES (Border External Security) Cluster of EU-funded projects lead by METICOS.
Crucially, the project explicitly “leverages the results and concepts implemented and tested for airport security within the H2020 FLYSEC and FP7 XP-DITE projects and for land border control in” the extremely controversial “H2020 iBorderCtrl project, and expand them into a multimodal border crossing risk-based security solution within a strong legal and ethics framework.”
Furthermore, the “TRESSPASS Consortium has been the Coordinator of all XP-DITE, iBorderCtrl and FLYSEC projects, and includes technical partners that bring on board an extensive experience in complex security projects and a roster of end users representing all three BCP modalities: air, sea and ground, including customs”.
Pilots and “future implementation” have been carefully planned and executed for the TRESSPASS project. In fact, deliverable D6.4 (framework for future implementation and validation of the TRESPASS solution including post-project) “offers a framework for the future implementation of the TRESSPASS Risk Based Border Management (RBBM) system”, including “examples of the three worked BCP cases from the TRESSPASS project, air, sea and land, respectively, and how implementation is highly dependent on the uniqueness of the BCP and the specificity of its needs.”
Deliverable D8.5 summarizes lessons learnt from pilots. It reads:
“For the testing and validation of this concept and tools, three pilots were planned to take place in different EU pilot BCP sites, each of them representing different use cases and modalities, namely the Dutch Pilot (air case, on threats from irregular migration), Polish Pilot (land case, and “focused on the application of the TRESSPASS system towards the specific threat scenarios of inbound illegal entries and cross-border crimes, such as smuggling illicit goods to EU carried and hidden in vehicles”) and the Greek Pilot (seacruise case, for checks at the port of Piraeus).”
Detailed analysis for each pilot is included in D8.5.
In the Dutch case, for example, the “use case for the pilot was narrowed down to detect travellers with the possible intention to irregular migration but in possession of valid travel documents and visa (if needed).” One might legitimately wonder what this “possible intention to irregular migration” might amount to.
Concerningly, “Web Intelligence was deployed to facilitate the information gathering of the involved passengers. For the needs of the scenario execution, the pilot owner facilitated the creation of synthetic social media (Facebook) profiles, which were populated according to the profile of the participating mala fide passenger personas. As soon as the travel details and social media information of each of those passengers was entered in TRA, WI was triggered to assess their digital footprint by calculating the risk indicators raised for their profiles” (according to “45 risk indicators”).
A worrying (and realistic) vision for the future of EU border checks is detailed:
“It is not realistic to suddenly replace the current systems and processes at once, it is too risky, technical unfeasible and could probably count on resistance of the current border guards and society. So, a gradual introduction is advised where increasing RBBM parts will be added as decision support in the current way of working. Starting with single extra indicators in the current primary border guard system and expanding this to complete profiles, experience and trust can be built and lead the way to seamless flows where bona fide assessed travelers can pass the border in self-service concepts without seeing any border guards.” “Trust” seems to be turned into a strategic tool for national security, according to this vision.
Similar concerns can be raised for the Polish pilot, for example for its “utilization of Facebook though data crawling, collection, and analysis in order to contribute to the calculation of risk indicators.” Additionally, in this case, interoperability with “legacy” EU databases — VIS, EES, SIS II, etc. — must also be addressed, as they fundamentally contribute to TRESSPASS’s risk analysis.
In fact, according to the deliverable, the “interoperability between TRESSPASS and these legacy systems aimed at providing useful information and alerts from passenger profile checks, regarding the construction of risk indicators. LSI implemented a list of simulated legacy information systems that leveraged the extraction of risk information. These simulated Legacy Information Systems were developed as Web Services.”
It is worth noting that such risk assessments will happen in real-time: “DRAS provided real-time risk assessment for each traveler based on continuously receiving traveler risk indicator data from Data Fusion Analytics (DFA) module, calculating traveler risk score per threat and overall risk score, using weight-based formulas, and providing traveler risk profile to the border guards through the Command-and-Control front-end user interface.”
TRESSPASS’s conflation of immigration, criminality, and terrorism is apparent from the very project description: “With regards to threats, this includes smuggling, irregular immigration, cross border crime, and terrorism, including threats to the transport itself (so, including e.g. aviation security – per the topic text)”.
The very idea behind the project — risk management should inform different screening procedures — is arguably (at least potentially) discriminatory and ultimately extremely contentious — even if adopted for example in the recent Migration Pact. This is all the more important given the crucial role assigned to a risk-based approach in the project: “Risk-based border management,” states the project’s Cordis page, “is about using border crossing points (BCPs) as a risk management measure that supports flow-, border- and national security. As such, border management is an essential element in a toolbox for mitigating a wide range of risks.”
The potential for discrimination becomes clear for example in the following passage:
“Risk-based approaches are typically used to select risk measures that are more proportional to the actual threat, while maintaining or even reducing the remaining risk: relaxed if possible, more stringent when needed. This implies that for people and goods that pose no significant threat, invasive checks at border crossing points can be limited. This should lead to less and shorter interruptions in the flow of people, more freedom for passengers and less additional personal data (w.r.t. data already collected before arriving at the border crossing point) that must be transferred at those points.”
A passage from deliverable D8.5 further illustrates how problematic this overall vision truly is:
“Risk-based concept is at the heart of the TRESSPASS system and as such, if it was introduced by EU Institutions, it could be applied with a certain amount of effort. Risk assessment is a continual process that takes place in each travel stage (pre-travel, approaching BCP and at BCP) collecting all the necessary data so that certain risk indicators, if any, match a profile that point to a threat category. Furthermore, knowledge at a pre-travel stage, allows border guards to act beforehand. Mitigating any side effect (for example bottlenecks at BCPs), in proportionality to the risk acceptance, may produce an overall positive outcome.” The notion that lies at the “heart” of TRESSPASS and is considered an “essential element” of its solutions “may” — or may not — “produce an overall positive outcome.” What if it doesn’t?
Conclusions on pilots in deliverable D8.5 are also telling of the overall mindset behind this and other BES Cluster projects. They argue that we had no alternatives to developing border surveillance technologies to truly solve border-related issues. “We need to adopt technologies such as biometric, data fusion and pattern-based threat prediction systems, so that we can improve detection, admonishes the deliverable. “We need to employ risk-based border management, so that we can increase the efficiency by focusing the attention on high-risk individuals.”
“For all these technologies (…) the appropriate training must be carried out, to make the right use of new technologies and to deliver on what they aim for”. However: “In general, with the large and massive movement of passengers such as the cruise, a tool that will make a significant contribution to cross-border control in terms of efficiency and speed must be adopted” (emphasis from the author, ed.).
Objective 3 in “Reporting” on Cordis adds another problematic layer: the notion “trustful passenger.” Both solutionist and securitarian, it reads: “Include passenger trust in risk management model and perform sensitivity analysis and optimization by taking into consideration a trustful passenger as a proactive and trustworthy source of voluntary information and determine the benefits from the development of such trust-based interaction between security system and passenger in order to optimize the performance of the system in terms of efficiency, cost reduction, and increased security.”
In reagard ethics assessments, several deliverables have been produced: D9.2 Project Baseline for Research Ethics; the Periodic Ethical Reports included in D9.3, D9.4 and D9.5; D9.6 Typology of ethical, legal and societal issues of risk based screening; D9.7 Framework for assessing direct ethical, legal and societal impact of risk based border screening concepts; D9.8 Updated framework for assessing direct ethical, legal and societal impact of risk based screening concepts; D9.9 Ethical Guidelines for Decision Makers.
It is however D8.5 that puts all these documents into perspective, i.e. in a trade-off with efficiency: “The three kinds of impelling principles of effectiveness, flow rate and efficiency should always be weighed against the anchoring principle of ethical compliance (D1.2 section 4.1 and D9.8)”. In this light, one might wonder what this means: “TRESSPASS attempted to build human values into the components and architecture by design.”
The Web Intelligence module is also problematic — and this is documented right in project documents. For example, again, in D8.5: “The idea behind Web Intelligence (WI) is that border guards might benefit from the publicly available data that travellers themselves published on the internet. This works by automatically detecting patterns of (combinations of) keywords in open sources. It is conceivable that this kind of technology in the future will also be capable of automatically interpreting images and videos.”
However, “significant operational and ethical questions remain. How easy is it for mala fide actors to evade or spoof WI? Should WI only be applied conditionally, or also unconditionally, i.e., for all travellers in a traveller flow? Should WI only contribute to risk indicators, or also to trust indicators? TRESSPASS ‘obtained’ the travellers’ social media references through the TRA. In reality, the USA obtains them as part of the visa application process. Which is the best way, or should it be part of (future) identity management systems or of the booking process (e.g., part of PNR)?”
On the last pages of D8.5 is admitted that ethics assessment procedures are perceived as an unnecessary burden that risk limiting technological achievements that would otherwise already be ready for deployment (and the market): “Although implementation of the framework was successful, it took considerable effort from the side of end-users and the ethics team to prepare and plan the pilots in accordance with it. The necessary documentation was sometimes perceived as an excessive burden. Also, the need to opt for alternative solutions to the use of real data from the public or re-use preexisting data owned by end-users needed in some cases extensive explanations and negotiations to be accepted by end-users and technology developers, as it was perceived as an unnecessary limitation to the technical and operational possibilities.”
A very useful Q&A was presented on the website, explaining personal data collected, surveillance systems adopted, how TRESSPASS respects “export control regulation” (what for?, ed.), “the typology of ethical legal and societal issues of risk-based border management,” and the ethical and legal compliance of research activities for TRESSPASS.
When it comes to the risk of misuse, deliverable D9.4 states: “The TRESSPASS technical manager suggested to create a list of TRESSPASS components for which the relevant CMs commit not to export to illiberal countries. As possible technology to be included in this list, MMCAT was mentioned. This suggestion will be followed up in the continuation of the project and it will be discussed which further technologies should be included in the list. For now, a commitment not to export the VTC (Video Tracking Component, ed.) component to illiberal regimes has been done by VV as it is stated in the related DPIA attached to D12.4.”
On MMCAT (the Multi modal communication analysis tool, ed.): “In February 2019, TNO and VV carried out a usability test for the MMCAT technology in Amsterdam. A few participants were asked to participate in a card game and answer some questions about the cards, sometimes telling the truth and sometimes lying. The scenes were video recorded and the video footage analysed in order to evaluate whether the MMCAT system could provide useful indications about the sincerity of the game participants” (D9.4, p. 17). All this lacks a solid scientific ground on which this might be based.
