FOLDOUT aims to develop a system to more effectively detect people and illegal activities in the contexts of border surveillance, in particular whenever “through-foliage detection” capabilities are needed to provide an “complete situation threat assessment”. It also promises to suggest the appropriate “reaction scenarios” for cases such as dense foliage, extreme climate or harsh environments.
The system will combine “various sensors and technologies” and “intelligently” fuse the data they’ll produce — including with “events detected by other sensors in the vicinity”. FOLDOUT promises that “by integrating data, such as vehicle traffic, from outside the immediate border area” even “pre-events can be detected and learned”.
“Machine learning tools” will also be developed “to continuously increase the systems detection and tracking capability”. Such a system is needed, assumes the project’s Cordis page in typical solutionist fashion, as “In the last years irregular migration has dramatically increased, and is no longer manageable with existing systems”, and therefore “Improved methods for border surveillance are necessary to ensure an effective and efficient EU border management.”
A detailed report on the project, its technology and tests (with infographics) can be found in deliverable D2.7, “Through-foliage detection, including in the outermost regions of the EU project Joint Evaluation and Recommendations report”. In it, it is interestingly highlighted that the same system applies to both irregular migration (in Europe) and illegal gold mining (in French Guyana).
Prediction is listed among the system’s functions.
According to the project’s Cordis page, “The technical concept of FOLDOUT is based on the combination of several sensor technologies on the ground and on special, high rising platforms with data fusion algorithms into a single, seamlessly integrated system”.
Given the objective of effectively detecting “critical events”, such as illegal border activities and lost persons through dense foliage, a set of “long range, multi-spectral, LIDAR and RADAR sensors” is also adopted. These will be augmented with “stratospheric platforms” deployed “to yield unobstructed field-of-view and unprecedented detection range” for large border areas — and with much more:
“Ground EM (radio transmitter detection), acoustic and seismic (movement) detectors will deliver complementary data where the vegetation is too dense to penetrate. The activities at the fringe of the foliage, such as suspicious car traffic, are monitored by conventional ground-based cameras and EOS sensors completed by satellite SAR data. Unmanned vehicles will be employed on demand to scan areas where sensor coverage is too low or data received are ambiguous”. Lastly, “Data processing algorithms based on machine learning will” also “be used to fuse and reliably interpret all data to derive alarms on the presence of persons or critical situations in the surveilled area. Reasoning methods will be used to filter unusual from usual behavior in the surveillance area”.
All of this should ideally happen in real-time, and be visualized through “a map-based graphic user interface with a standardised symbology to observe, track and react with maximum efficiency”.
A synergy was established between the FOLDOUT and BORDERUAS projects, focussed on “clustering actions” and on “highlighting common objectives and potential areas of cooperation”.
More concretely, FOLDOUT’s “Multimodal Fusion Architecture for Sensor Applications” is providing the basis for further improvements within the more recent Horizon Europe project EURMARS.
In addition, deliverable D11.3 (Dissemination and Communication Plan) identifies some “Related H2020 Projects”. These include SMILE, iBorderCtrl, PROTECT, PERSONA and TRESSPASS.
At the time of writing, the FOLDOUT Consortium was however still “waiting for the feedback from other projects”, in terms of proposed forms of cooperation, including for “relevant Events”.
No further updates or details could be found.
Through “A two year pilot in Bulgaria and demonstrators in Greece, Finland and French Guiana (lasting between 2 and 4 months, cfr. D3.5, ndr)”, FOLDOUT promises to “provide fundamental enhancements in the domain of border surveillance and improved search & rescue scenarios.”
These will consist of “simulated operational scenarios, where only members of the consortium organizations took part to verify that the system performs as expected”. “Vulnerable groups” — “Elderly, Minors, Victims/Survivors of trafficking, people with disabilities (…), pregnant women (…) and asylum seekers (…)” — were not included, according to deliverable D3.5; they will however be subject to the system, once and if implemented.
Also, YouTube videos have been published concerning
1) The 2022 Finland Demo, in which a dog and a person are identified and tracked (interestingly, D3.5, detailing the pilot conditions, argues that “it has been clarified between the partners and to the EC that no biometric data was going to be/have been processed as part of FOLDOUT. No tracking of individuals has taken place”), whereas a vehicle and a group of people in hiding are detected; a van is also tracked, and subject to thermal screening together with the surroundings. A new drone “potentially” used as “a FOLDOUT sensor platform” is shown in a second video;
2) The 2021 Greek Demo, and The Bulgarian Demo (also held in 2021), whose video includes an interesting few seconds on the system’s combined view from RGB-IR Cameras, Thermal Cameras, and “Situation Awareness”. A banner for a “Virtual Border Test” on June 14-18, 2021 was also shown.
Deliverable 2.7 details all testing sites and dates: Stara Zagora, Bulgaria (11.11.2019 – 15.11.2019; a further test was conducted on 31.05.2021-04.06.2021”); Greece (19.07.2021 – 24.07.2021 and 27.07.2021 – 30.07.2021); French Guiana (04.10.2021 – 15.10.2021); and Finland (24.1.2022 – 28.1.2022).
The project is clearly meant to primarily benefit border guards: “The FOLDOUT platform will assist border guards by providing prompt detection of illegal activity at borders and trace the movement and routes prior to arrival in border areas. (…) FOLDOUT will make the tasks of Border Guards simpler and faster by combining events from various sensors (…)” — and yet, in the Dissemination and Communication Plan (D11.3) the slogan is “Through foliage detection for EU citizens safety”.
The project’s main page claims that “FOLDOUT delivers technical solutions that are in-line with EU Legal-Ethical-Privacy rules and regulations. Technology impact assessment anticipates legal and privacy concerns. The collection, processing and production of the data are compliant with GDPR”, but provides scant public evidence concerning the details of any of these checks — as, even two whole years after completion, only six project deliverables are available to the public (and only a couple concerns privacy and ethics considerations).
Also, D11.3 clearly marks the development of innovative technologies (and therefore the driving role of innovation) as the utmost priority, thus satisfying pillars of EU border security policy such as “Showcasing Europe’s leadership in security technologies” and “Creating a European Border Surveillance Technology with growth potential”. And yet (or consequently, actually), “FOLDOUT has already been presented at 17 EU/International security events and dissemination material in the form of presentations (virtual or real) combined with website, posters, flyers, videos produced and distributed”.
A “privacy impact assessment of the project outcomes” (which considers “algorithmic fairness”, even though only as a GDPR requirement, D3.4) and some “ethical guidelines” (D3.5) are provided among the few above-mentioned deliverables on Cordis.
They can however hardly be enough to address all issues arising from such an invasive surveillance apparatus — and are certainly not enough in terms of providing actual transparency on its operations and impact.
Interestingly, D3.5 asks an uncomfortable question: what happens if the system ends up in the wrong hands, and is used by criminals?
The same deliverable then illustrates the societal impact and risks derived from the use of FOLDOUT. For example, it may infringe on freedom of movement: “The technologies such as those used in FOLDOUT provide the means to monitor and categorize those who move, and even create ‘soft checkpoints’ away from critical areas such as border crossings (Razac, 2009). These soft checkpoints can not only limit the freedom of movement in areas where it should prevail but can also have other social impacts in human rights due to function creep”.
“Autonomy” may also be affected, as well as “justice” (and therefore fairness).
