Stories

For trans people, online transitioning can be a nightmare

Just a formality? Updating one’s online identity can be like running the gauntlet and a flawed data architecture only makes it worse for the transgender community.

Image generators are trying to hide their biases – and they make them worse

In the run-up to the EU elections, AlgorithmWatch has investigated which election-related images can be generated by popular AI systems. Two of the largest providers don’t adhere to security measures they have announced themselves recently.

“All rise for the honorable AI”: algorithmic management in Polish electronic courts

Polish courts are using algorithms to support their decision-making process, e.g. for evaluating cases or issuing resolutions. Some see AI as a game changer, but the lack of a critical assessment and transparency as well as the impact on judges’ independence and fairness are reasons for concern.

Campaign: ADM and People on the Move

The automated Fortress Europe: No place for human rights

29,000 people have died in the Mediterranean over the past ten years while trying to reach the EU. You would think that the EU wanted this tragedy to stop and scientists across Europe were working feverishly on making this happen with the latest technology. The opposite is the case: With the help of so-called Artificial Intelligence, digital border walls are being raised, financed with taxpayers' money.

Italy introduces entirely automated public tenders

The neofascist government led by Brothers of Italy passed a law that encourages public institutions to entirely automate procurements, from the definition of needs to the selection of winning bids. The consequences are uncertain as such systems have yet to be implemented, but small communities will most definitely lose power and opacity will reign.

Spanish inmates not to be automatically monitored in fear of AI Act

Spanish region Catalonia’s government approved the use of an Artificial Intelligence-based software to monitor inmates and interpret their behavior. Partially funded by the European Union, the system was to be implemented at the Mas d'Enric prison near Tarragona, a city south of Barcelona, and extended to other regional prisons. Ultimately, it wasn't.

Austria’s social security invests over €50m in AI – just  for bookkeeping?

The company that manages the technical infrastructure of Austria’s health insurance system passed a massive tender for AI services. The plan is shrouded in secrecy, even though the company’s track record urges caution.

Fighting in the dark: How Europeans push back against rogue AI

Automated systems go astray, they contribute to child sexual violence, deny people their social benefits, or block organizations' online presence. The affected people often feel helpless when their rights are violated, but some are taking up the fight while current laws fail to protect the victims.

How not to: We failed at analyzing public discourse on AI with ChatGPT

We wanted to collect data on how newspapers wrote about AI. Swayed by the sirens of OpenAI, we thought that using a Large Language Model was the best way to do such a massive text analysis. Alas, we failed. But LLMs are probably not to blame.

Work inside the machine: How pre-saves and algorithmic marketing turn musicians into influencers

Streaming platforms allow users to add upcoming tracks to their playlists, in order to listen to them as soon as they are released. While this sounds harmless, it changed the habits of independent musicians, who feel they have to adapt to yet another algorithm.

Greece is planning a €40m automated surveillance system at borders with North Macedonia and Albania

The European Commission wants Greece to build an automated wall to prevent some people from leaving the country. Locals are not enthusiastic, but their opinion counts for little.

The year we waited for action: 2023 in review

Exactly one year ago, I wrote that automated systems might be regulated for good in 2023. This was too optimistic. Not only did European institutions fail to pass the AI Act, even in its watered-down version; the rise of generative models brought us to a new level of danger.

Plant-identifying apps: good for amateurs, bad for students

Apps that automatically identify plants have become immensely popular among amateur botanists. While they might help them in their hobby, they also made inroads among students and professionals, with potentially serious effects.

Platform regulation

Not a solution: Meta’s new AI system to contain discriminatory ads

Meta has deployed a new AI system on Facebook and Instagram to fix its algorithmic bias problem for housing ads in the US. But it’s probably more band-aid than AI fairness solution. Gaps in Meta’s compliance report make it difficult to verify if the system is working as intended, which may preview what’s to come from Big Tech compliance reporting in the EU.

Some image generators produce more problematic stereotypes than others, but all fail at diversity

Automated image generators are often accused of spreading harmful stereotypes, but studies usually only look at MidJourney. Other tools make serious efforts to increase diversity in their output, but effective remedies remain elusive.

Algorithmic blood donations in Ukraine

On paper, Donor.ua solves many of the inefficiencies of blood donorship in Ukraine. It connects people willing to donate with those in need thanks to a matching algorithm. But implementation proves difficult, and the war is not the only reason for it.

Food delivery service Glovo: tracking riders’ private location and other infringements

A recent investigation by Tracking Exposed shows that Glovo’s subsidiary in Italy, Foodinho, registers couriers’ off-shift location and shares it with unauthorized parties. The delivery app provider has also been found to have created a “hidden” credit score for their riders.

Spain under shock as schoolboys create fake nudes using generative models

In a small Spanish town, several schoolboys used generative models to create fake nudes of their fellow pupils. Police, prosecutors, and parents are at a loss on how to pursue a case that shows, once again, that women are the main victims of deepfakes.

Generative Artificial Intelligence is slowly entering children’s lives

A leak from Amazon gave us a glimpse of what they have planned for youngsters - and parents. How much personal data do we (or should we) accept handing over when it comes to our children?

Automated navigation systems are still wreaking havoc on small towns’ streets

Apps like Google and Waze are redirecting traffic to secondary roads that are not equipped to handle the traffic, disrupting their infrastructure. Small cities have little capacity to change this.

Algorithmic Accountability Reporting

Peeking into the Black Box

Welfare fraud scoring, predictive policing, or ChatGPT: Lawmakers and government officials around the world are increasingly relying on algorithms, and most of them are completely opaque. Algorithmic Accountability Reporting takes a closer look at how they work and the effects they have. But only very few media outlets conduct such reporting. Why?

200 students failed their exams. Automated proctoring could be to blame, but doubts remain 

In Spain, 200 students of the International University of La Rioja failed their exams. Some blame a glitch in the proctoring software, but it might have been a change in the system’s rules. University officials gave contradictory explanations, leaving students to fight against bureaucracy and the assessment of a machine.

AI-powered surveillance

In Mannheim, an automated system reports hugs to the police

Mannheim, a large city on the Rhine, deployed a video system that claims to automatically detect physical violence in some streets. It can confuse hugging with strangling, and it is unclear whether it can actually prevent violence.

The Automated Hunt for Cybergroomers

Algorithms have been developed to help track down cybergroomers, who stalk minors online. But the reliability of automated systems is controversial – and they can even criminalize children and teens.

ChatGPT-like models boom, but small languages remain in shadows

A lack of source material, investment, and commercial prioritization are all holding back the development of generative models and automated moderation for languages spoken in smaller countries and regions.