Twitter’s algorithmic bias bug bounty could be the way forward, if regulators step in

Twitter opened its image cropping algorithm and gave prizes to people who could find biases in it. While interesting in itself, the program mostly reveals the impotence of regulators.

On 30 July, Twitter launched the first algorithmic bias bug bounty program. They opened the code of their image cropping algorithm and asked for people – anyone could participate – to look for new biases. The algorithm in question had been accused of racist bias in September 2020, after a user showed that an image of Barack Obama and a white US politician systematically cropped out the former president.

Bug bounty programs are common in computer security. Most large software firms run them, offering monetary rewards for new bugs – the more severe the security issue, the higher the reward. One of the conditions for payment, of course, is that the bug not be made public until the software vendor had the time to fix it.

Twitter let the bug bounty run for a week. At the end, they received over 30 contributions from individuals or teams, including one high school student.

Participants showed that the algorithm cropped out people with white hair, as well as non-Latin scripts from pictures, or that it automatically focused on the last image in a comic strip, thereby ruining the suspense for the user. The winner of the bug bounty program built a tool that optimizes faces to increase the likelihood that they will not be cropped out (it turns out that younger faces with a light skin tone and stereotypically feminine traits perform better).

Twitter’s program is an unprecedented experiment in openness. The cropping algorithm is copiously documented and very easy to run (even though it seems biased against AlgorithmWatch’s logo). It is likely that more researchers will use it in the future to report new findings and develop new methodologies.

The “saliency map” of Twitter’s image cropping algorithm (left), and the actual crop (right).

Although Twitter should be lauded for this experiment, it remains far short of addressing the harms caused by algorithmic bias.

First, Twitter only opened a relatively unimportant algorithm, to which users already had unfettered, if unpractical, access (anyone could create a dummy Twitter account, automate the posting of pictures and check how they were cropped). Opening the timeline algorithm, or the one matching users to ads, would be something else entirely. Perhaps Twitter has plans to do so in the future. (Twitter refused to answer our questions.)

More prosaically, the bug bounty offered only 7,000 dollars in prizes, shared between five winners. That translates to 1,400 dollars per winner, and to 0 dollar for the roughly 25 participants who might have identified a new bias but were not selected by the jury. “It would have taken us a year” to identify all these biases, said Kendra Clarke of Twitter’s ethics team during the awards ceremony. Considering that 7,000 dollars is about two weeks’ worth of a software engineer’s wages, it is easy to see who benefited most from the program.

In contrast, the average payout for a security-related bug is over 10,000 dollars, an order of magnitude higher. Unless the financial reward increases to the point that bounty hunters can live from their findings, it is highly unlikely that algorithmic bias bug bounties will become as successful as their model in security.

Software companies pay larger sums for security bug bounties because they have an incentive to do so. Security might be a selling point for customers, and firms have regulatory obligations to disclose data breaches which, in turn, encourages them not to have data breaches.

No such incentives exist for algorithmic bias. On the one hand, the population affected by algorithmic bias are not paying customers, as Deborah Raji, a research fellow in algorithmic harms for the Mozilla Foundation, told ZDNetearlier this year.

On the other hand, although discrimination is prohibited by law, the agencies in charge of enforcing this legislation are underfunded and wholly unable to assess algorithmic bias. Things are changing, as some Member States of the European Union build their algorithmic auditing capacity. In France, PeREN, an “expertise group for digital regulation”, brings together about twenty people, including software engineers, to support regulatory bodies across the country.

The pace of the regulators’ capacity building remains much slower than the deployment of new, and potentially biased, algorithms. Bug bounty programs could help identify and fix some of these biases. But regulators must increase the pressure on software vendors and operators until they have an incentive to provide bounty hunters with decent rewards and plenty of algorithms to investigate.

Nicolas Kayser-Bril


Photo: Julia Bornkessel, CC BY 4.0
Nicolas is data journalist and working for AlgorithmWatch as a reporter. He pioneered new forms of journalism in France and in Europe and is one of the leading experts on data journalism. He regularly speaks at international conferences, teaches journalism in French journalism schools and gives training sessions in newsrooms. A self-taught journalist and developer (and a graduate in Economics), he started by doing small interactive, data-driven applications for Le Monde in Paris in 2009. He then built the data journalism team at OWNI in 2010 before co-founding and managed Journalism++ from 2011 to 2017. Nicolas is also one of the main contributors to the Datajournalism Handbook, the reference book for the popularization of data journalism worldwide.