An AI slop farm stole my identity

ArchZine. I start most workdays by googling myself. Some might call it vanity, I consider it digital hygiene. On 23 August, I was startled to find a profile under my name on a website I'd never heard of: ArchZine, an online magazine. It featured my photograph, an accurate biography, a link to my Instagram and even an email address, nikolas@archzine.fr! The articles forcibly put under my pen were, quite obviously, automatically generated, ranging from fabricated current affairs to horoscopes.

It was not the first time I was the object of AI-generated fakes. I'd already come across counterfeit interviews of me on YouTube, for instance, but this felt more serious. Granted, these weren't genuine deepfakes, and the harm they caused was trivial compared to non-consensual AI-generated nudes. Still, I felt I had to do something.

Complaint mechanisms. ArchZine is owned by a small Bulgarian company. I immediately contacted them, demanding that they stop using my identity. They ignored my e-mail. I then reached out to Bing and Google, asking for the articles falsely attributed to me to be de-indexed. Bing complied, but Google refused. The page was "obviously related to my current job", I was told in an email. I was told I would need to prove that I did not, in fact, work at ArchZine, Google told me. How exactly was I supposed to prove a negative - a notoriously impossible task? I emailed Google again, but they dodged the question.

I then contacted the data protection authority of Berlin, where I live. They dutifully forwarded my complaint to their Bulgarian counterpart. I haven't heard of them since.

The slop economy. ArchZine is a textbook example of an AI slop farm. It churns out vast quantities of automatically generated articles and images, drawing traffic from aggregators and platforms such as Pinterest. I assume that they were interested in my identity in order to be recognized as a journalistic endeavour and be featured on Google News. Alongside mine, they also stole the identity of another French investigative journalist. I could identify eight other people – influencers ranging from a chef to a hairdresser, in France and Germany - whose identities had likewise been stolen.

Any human looking at the content would immediately dismiss it as AI slop. But in an economy where every automated system feeds the next, real damage becomes almost inevitable. The articles attributed to me on ArchZine, once picked up by Google News, appeared on my Muckrack profile, an automated aggregator for journalists. From there, if you ask ChatGPT about me – tada – you'll learn that I work for a Bulgarian content farm.

Criminal law. I contacted specialist lawyers in Paris, who sent a letter to ArchZine's French office reminding them of the penalties they faced: from identity theft (punishable by 1 year in prison) to the illegal collection of personal data (5 years). My profile was promptly renamed. Perhaps out of spite, ArchZine replaced me with yet another stolen identity. This time, that of a lawyer.

Despite this egregious behavior, alerting the authorities would have required fresh resources I simply didn't have. In the end, I had to pay a considerable sum on legal fees – roughly equivalent to two weeks' wages at the minimum rate – to protect my reputation. Even though my story is a fairly simple case of defamation, already covered in criminal law (in Germany, it's been sitting at § 187 StBG since at least 1870), I can't help but note that data protection legislation such as the GDPR did not help much. I alerted all the victims I could identify. Yet, two months on, many fake profiles remain online.

Identity theft for the purpose of reputation harvesting is still new, the Berlin data protection authority told me. However, it has the potential to grow wildly in sectors such as journalism or academia. Scholarship is already rife with fabricated citations. Predatory journals attributing fake articles to existing, honest scholars would have little to fear.

This is an excerpt from the Automated Society newsletter, a bi-weekly round up of news in automated decision-making in Europe. Subscribe here.