AI surveillance rumors: gay adult content creators face sanctions

Early last month, many gay fetish accounts were charged for distributing online porn to minors, a criminal offense in Germany. Many suspect an automated tool, but no one knows for sure.

Story

4 April 2023

#lgbtq #twitter

Jake - not his real name - has a fetish for bodysuits, sneakers, and latex. "I'm not at events or live meets,” he told me. In his gearfetish Twitter bubble, he takes pictures of his sneakers and his outfits (enjoying the March sunshine in a sporty tracksuit with a rubber body suit underneath). He connects with guys who like the same things. One acts as a virtual fitness coach, monitoring his swimming times. 

Reid has also met people via dressing up in tactical wear that turns him on ("turning into gear" with a one-piece leather suit and a helmet, for example) and sharing pictures online. "Many of us connect to each other through Twitter, since we are a little niche, it is hard to find like-minded folks through other venues," he writes. At a real life gearfetish event in the Netherlands a few years back, Reid, who went alone, recalls the best moment of the evening being in a hug on a sofa with one person he'd already met online and new acquaintances. 

Now in the past month, a bunch of accounts integral to the German bondage-fetish scene (and beyond) have gone private and people are spooked  – due to a wave of legal sanctions against gay adult content creators, which has sparked rumors about AI surveillance.

Section 184

Last month, Fabienne Freymadl - a sex worker and, in the face of Germany's ongoing crackdown on porn, online youth protection officer - received messages from several gay erotical creators. Some had just been charged under section 184 of the German criminal code, which prohibits distributing pornography on social media without age-checks to make sure viewers are over 18. Others were told by Twitter that they’d been reported for posting porn. Then their accounts were blocked in Germany. 

Section 184 has been around for a while. "It happens regularly that accounts of sex workers or people presumed to be sex workers are suspended out of the blue," says Freymadl. "But I have not yet heard of such a concentrated action, which only impacts one kind of pornography [gay kink]." 

The Berlin state media authority, which monitors social media for violations of section 184 (as well as hate speech and anti-constitutional symbols) reported 104 cases of porn to the public prosecutor’s office in March. Freymadl has heard of criminal charges in other states such as Lower Saxony. She says that there also are “a lot of geo-blocked accounts,” meaning that their tweets cannot be viewed by German IP addresses.

In early March, rumors began to spread that KIVI, a tool that uses Machine Learning, was behind the sanctions. Since 2022, KIVI is used by German state media authorities to crawl through social media profiles to flag posts, according to certain keywords and geolocation. Employees decide if the post is a violation and whether to alert the public prosecutor.

In Berlin, media authority press spokeswoman Anneke Pass says they did not use KIVI to find the 104 accounts. Freymadl speculates that the explanation is not necessarily technical: "It could also be that a fundamentally homophobic fool decided to report a bunch of accounts."

German state media authorities tell Twitter to block accounts

Heiner - not his real name - lives in western Germany and uses Twitter to meet people to have fun and make gay sex videos with, and to get feedback from his followers. "For me it's a hobby, and not about money," he says. When Twitter geo-blocked his account for German audiences in early March, he was surprised: “I thought I wasn’t doing anything illegal, because so many people are doing it and Twitter has policies. I mean there are tens of thousands of people posting porn.”  

Another question he has: “German media authorities are not law enforcement, do they just ask Twitter to shut us down?"

In March, the state media authority in North Rhine-Westphalia, headed by "fetish for regulation"-driven director Tobias Schmid, did not report any Twitter porn cases to the public prosecutor's office. Instead – to save time and to find out if their relationship with the platform has been impacted by Elon Musk's takeover – they reported the accounts to Twitter directly, which then blocked the accounts in Germany.

In the North Rhine-Westphalia office, there are around five students working part-time to find guilty accounts, often using the KIVI tool (which is more effective at flagging porn than hate speech). If they score a gay kink account, for example, they look into that person’s friends and for the images they have liked. 

At the moment, Heiner’s friends from other countries can still see his profile and he can send direct messages to Germany-based users that he wants to collaborate with. “If this is the law in Germany, then I would prefer not to be visible here,” Heiner says, “I can still have my fun with the rest of the world.”

Self-censorship 

On 10 March, the account Rope_Phantasies – a longtime bondage instructor – shared an image of a megaphone, warning that the German media authorities were cracking down on independent adult content creators from Germany. 

"I noticed that many people were deleting their posts, making their profiles private or deleting them, which I think is a pity," Jake writes. He read several articles about KIVI and was shocked. He went through 800 of his past tweets and deleted 30. Now he's back online with the sportsgear pics. Rumors continue to circulate that KIVI is targeting LGBTQ users, that even liking a smutty post can get you prosecuted and that making your profile private doesn't help. “I'm also feeling unsettled, because even being suspected of pornography can have negative consequences for my job.” 

Neither Jake nor Reid know any fetish accounts that were charged or blocked. But several prominent accounts have now gone private, deleted past content, or are moving to Instagram. Some are “a real loss for the bondage-fetish scene,” according to Jake.

For now, Reid is opting to change his aesthetics. "Personally, I made the decision to focus on more sensual, artistic expressions of my kinks without any nudity and such," he writes. 

How social media regulates gay kink

"Twitter is at this point kind of the last big platform for this community," writes Reid.

In 2018, Tumblr – a former meeting point for people to explore their sexuality – decided to remove all “not safe for work” and explicit content, after child pornography was found on its platform (the policy was criticized as having more to do with advertising sales than wanting to protect users.) Porn sites like xTube, where gay fetishists and kinksters used to upload their videos, have been shut down as well. (When PornHub got struck with articles saying that they do not do enough to combat illegal content like revenge porn, they responded by deleting all private user uploaded content before pulling the plug on sites like xTube completely.) 

For now, Reid writes, "the LGBTQ kink community found their home on Twitter, since Twitter has always been very liberal about porngraphic content."

But given the events of the past weeks, he adds, "we definitely have to be a lot more careful and self-censor what we post and share." There are Mastodon instances, Telegram groups, and even some dedicated websites. But "nothing ever managed to really establish itself as a viable alternative and [leaving Twitter] would just splinter this already small community into even more little bubbles all over."

Meanwhile, Jake – whose last pic involves another sports-themed number – says he will continue to like “a lot” of gay porn, adding “If I get a problem, then so should half of Twitter.” He writes, "I have decided to stay cool and continue. I produce fetish content and sometimes one might guess what lies beneath the gear."

Did you like this story?

Every two weeks, our newsletter Automated Society delves into the unreported ways automated systems affect society and the world around you. Subscribe now to receive the next issue in your inbox!

Get the briefing on how automated systems impact real people, in Europe and beyond, every two weeks, for free.

For more detailed information, please refer to our privacy policy.