AI Generated Content

Related terms:
See also:
Click here for related articles on Fanlore.

AI has become an increasingly discussed and controversial tool in the realm of art and creativity. Many artists, writers, and other creators have expressed their concerns regarding its use in generating content and art, whilst others have voiced interest and positive thoughts on its integration.

The primary criticisms of AI in this context include its reliance on plagiarising and exploiting the works of creators (lack of clarity and transparency about the source of the data, unethical sources of data to train the AI model which includes content people has created for free to develop a product that will allow developers to profit from), its use as an alternative to paying creators decent wages for their work, its dystopian-like place in late stage capitalism, and the perceived soullessness of the content generated.

Impact of AI models on fandom

The emergence of AI models has had a big impact on fandom, leading, among other things, to notable changes in the terms of service of art platforms. While AI tools have been available for some time, their popularity experienced a significant surge in 2022. Users began exploring the capabilities of AI models, resulting in an influx of AI-generated content across various websites which, after the initial curiosity and novelty died out, wasn't well received by the userbase. The rapid growth of this trend presented new challenges for websites and online communities, prompting them to carefully consider how they adapt their policies.

As generative AI continues to develop, some fans of different media have become increasingly uneasy about the possibility of AI-generated content being added into or used in media such as TV, films, books, and video games. This has led to speculation and accusations of AI use, some of which has been confirmed or denied. For example, some people felt that the Amazon Prime show Citadel had a script written by AI; Pocketpair, the team behind Palworld had openly used AI on a previous game and so accusations about AI use in the creation of Palworld were wide-spread despite Pocketpair's denial of its use; and Marvel's Secret Invasion TV series had AI used in the trailer, with created backlash from fans. Similar suspicions and accusations have also occurred regarding fanworks, causing hurt and frustration for some authors targeted by the comments. In 2023, a numbers of Archive of Our Own authors had fanworks hit by a comment bot, which was leaving comments on thousands of works claiming the work had been detected as AI generated.

Pick it up! banner made for #NOMORE無断生成AI by ACE-kow. (2024)

Different online platforms used to host fanworks take different stances against AI-generated works. Some conventions of fanartists banned AI-generated works in their venues.

In March 2024, a series of tags were created on Twitter by Japanese artists and voice actors to protest against AI-generated content and scraping. The primary tag used was #NOMORE無断生成AI. People employed it to voice their stance against AI, encourage others to learn how to draw, share any instances of AI usage by companies, and highlight flaws in AI-generated art being misrepresented as traditionally made art.

In June 2024, a petition was started in Change.org Japan urging government agents and political parties to create laws to protect creators from Generative AI. An English version can be found here.

Platform-Specific Stances

AO3/OTW

In May 2023, the Organization for Transformative Works (OTW) took the position that if fans use AIs to produce fanfiction, these are "a type of work that is within our mandate to preserve". This position has been controversial within fandom, with many fans expressing their dismay at this stance and asserting that using generative AI to create fanworks is akin to plagiarism, due to the fact that fanworks are known to have formed part of the dataset used by generative AI tools. However, other fans have expressed sympathy with the OTW's stance and pointed out that the technology does not exist to reliably detect AI-generated fanworks, meaning that any ban would be impossible to enforce and would inevitably ensnare many legitimate fanworks.

Pixiv

Pixiv have acknowledged the impact of AI models on their website. In response, they made changes to their terms of service to address the presence of AI-generated content. Users now have to tag AI-generated content and have the option to customize their settings to exclude AI-generated works from their browsing experience.

Pixiv banned AI art in paid commissions on their platform.

DeviantArt

In 2022 DeviantArt released "Dream up," an AI model designed to generate images from user prompts. The AI model was not well received by the userbase. One of the main reasons that upset people was the fact that "Dream up" was a paid service, with only five free prompts given for free to each user like a trial. Additionally, there were suspicions that the AI model may have been trained using images from DeviantArt without proper attribution or consent. This is because, when first released, Dream up participation was automatically turned on for all art, meaning all art on DeviantART was automatically opted-in to be scraped by the program. This prompted a large number of users to remove their art from the website and close their accounts. However, DeviantART quickly changed their policy to automatically opt-out all DeviantART art from the AI's training so that artists could opt-in manually instead. By then the damage may have been done.

Large, popular AI content farms appeared on DeviantART in the mid 2023s, with interconnected users joining at the same time and selling massive amounts of AI generated art and adoptables for cheap. On ToyHouse, artists began making lists of these content farms to be blocked, which can be seen in a PSA thread tracking them.

SquidgeWorld Archive

Following the AO3 scraping controversy, SquidgeWorld Archive clarified its stance on AI scraping and AI-produced content, stating that AI-generated fanworks are "not supported, wanted, or allowed in the archive", with the exception of fragments used in meta essays.

Danbooru

Danbooru banned AI-generated works, while allowing AI-assisted works with "high-quality human retouching."

Twitter / X

X has announced a update of its Terms and Conditions, which allows the site to use user contents to train generative AI.

Chinese platforms

Chinese platforms, such as Bilibili, generally allow AI-generated content, train their generative AIs with user contents, and promote generative AI. In Bilibili, AI-generated songs using professional singers' voices without their permission (such as Stefanie Sun) were popular in 2023. See also AI Generated Content and Chinese-Speaking Fandom.

Fandom Community Approaches

Gift exchanges, fanweeks and other similar events have started to include AI-generated content in the list of things that aren't allowed as entries.

After users started expressing their annoyance at the saturation of AI-generated works on their feeds, certain subreddits on Reddit took measures to gauge public opinion on the matter. These communities conducted polls and surveys to gather people's opinions and explore the possibility of implementing bans or restrictions on AI-generated content within their respective spaces.

Gay Sex Cats

In September 2023, an AI generated image showing six kittens with a hidden message that said "gay sex" generated a huge amount of discourse on Tumblr and other platforms, about whether or not AI is acceptable in some cases, or whether or not hatred for AI generated content is blown out of proportion when it's used for memes.

Using AI to produce fanworks

Fanart

Initially, sites like NeuralBlender and Craiyon were used by fans to generate fandom related images that were oftentimes humorous in nature. However as AI and its mechanisms became more well known, opinion began to swerve.

In the mid-2022, users in social media platforms began voicing their frustration and annoyance regarding the proliferation of AI-generated images. Initially perceived as intriguing and novel (and a new source of memes), the growing prevalence of these low-quality, machine-generated artworks soon led to a sense of fatigue among the user base, as these outputs began to dominate their feeds. Amidst the discussions, some users expressed that while AI models may possess a certain allure, they could never truly replace the creative abilities of human artists.

Another common criticism of AI-generated fannish art revolves around the lack of fidelity to the original characters. Many fans have expressed disappointment and frustration with the machine-generated images, noting that they often fail to capture the essence and distinctive features of beloved characters. AI models often struggle to accurately represent the nuances of facial expressions, body language, and unique characteristics that make characters instantly recognizable.

ChatGPT (or specifically it's generative art model DALL-E) will reject requests to create fanart, often citing copyright issues. Many users have attempted to bypass this restriction and "trick" the model into creating fan art, like pretending that the copyright has expired, or they are OpenAI developers testing the model's vulnerabilities. This is typically known as "prompt injection", a well-known issue with AI models, where they can be tricked into breaking their own rules. A viral example of this was an incident where GPT rejected making a Sonic the Hedgehog artwork, but when asked for a "blue bipedal hedgehog", it produced a Sonic artwork without issue.

Identifying AI-Generated Art

Since the increase in AI-generated fan art, artists have been creating guides to help identify AI-produced works, as the technology improves and imperfections become less noticeable over time. Some individuals download AI-generated art and mark any details that seem out of place, such as extra fingers or blurred shapes, with red circles.

Some of the most common indicators of AI-generated art include weird hands (such as extra fingers, missing fingers, or fingers merging with each other), hair strands blending into clothing, asymmetry, odd shading, absence of small details like buttons, and distorted intricate details such as ribbons, umbrellas, or food. Other signs may include inconsistent lighting, unnatural poses, changes in style and backgrounds that do not seamlessly integrate with the main subjects.

There is an AI art detector available called Maybe's AI Art Detector, but it does not function reliably and cannot be fully trusted, as it often incorrectly identifies AI art as human-made or vice versa.

• How to identify scarily-accurate ai generated anime art from hand-drawn anime art (guide) by Coilovercat
• Just tips to identify AI art by Justsyl

AI-Generated Examples

• 

  Image generated in DALL-E 2, using the prompt "Ladybug and Chat Noir from Miraculous Ladybug as adults". Example of how inaccurate AI Models can be, both in representing the characters and the prompt itself. While technically true, DALL-E 2 portrayed the characters as elderly persons.

• 

  Set of 4 images of Kujo Jotaro from Jojo's Bizarre Adventure generated with NovelAi. Prompt: "photorealstic, jojo_no_kimyou_na_bouken kujo_jotaro 1boy blue_eyes blue_hair chain hat male_focus solo". The AI model cannot accurately pinpoint the elements on the cap, so it places random shapes there. In some of the generations, the chain is missing.

• 

  Example of AI Generated image with inconsistencies highlighted. Prompt and model unknown.

Links

• AI-Generated Images tag at AO3

Fanfiction

Text generative AI programs such as ChatGPT have been used to create many text-based fanworks. These works are often posted with little or no editing, though many have also been edited to varying degrees by their posters. ChatGPT has also been used as a translation tool for many authors.

An early example might be Harry Potter and the Portrait of what Looked Like a Large Pile of Ash, a Harry Potter based story which was produced with the help of predictive text input in 2017. However, even though the work was sometimes said to be created by "AI", it was before the introduction of large language models which enabled more powerful tools like ChatGPT.

AI-Generated Examples

• "You are the best Valentine ever!" made in an undisclosed AI model by Aleee_rv for the ship Rhadaval. Prompt: Lord Rhadamanthys was waiting for his dear Valentine, alone in his apartments.

SO. I was playing with an AI and I gave it a single line of prompt (the first one you read.)

Then all by itself it came up with this short story. Somehow it knew that Rhada = Dragon, rather impressive! And even that he likes to drink? Tho that might have been a coincidence I guess given how a generic thing that is. ANyway, it turned out as something entirely stupid and funny borderlining nonsense XD SO I NEEDED TO SAVE IT AND SHARE IT SOMEWHERE XD Rhada is OOC but he feels like a dork semi-drunken version of himself XD I kept everything as the AI came up with, just made some basic editing for grammar and such.

Enjoy, I guess? XD I'm sorry for this cursed thing XD I hope it makes you laugh as much as me XD

• Tea Date Bliss made in ChatGPT by Rhov for the ship Ereri. ChatGPT test made for April's Fools.

For April Fool's Day, I decided to let ChatGPT write some fanfics for a few of my favorite couples. This one is for Eren and Levi. I'll admit, I had to splice three attempts together and had to keep feeding it extra prompts. Still, not bad for AI. Even the title was generated.

Prompt: Write a story based on the Attack on Titan anime. Eren and Levi are in an established relationship. Eren takes Levi to a posh tea shop on a date so Levi can relax. Eren loves to watch how elegant Levi looks drinking from a porcelain teacup. The cream from the pastry puff gets on Eren's nose. Levi teases that he's a messy brat, swiped off the cream, and sucks it off his finger, leading to a kiss. Additional prompts: - change to Levi knowing a lot about tea and Eren being unsure, so Levi suggests honey sweetened chai latte. (I was drinking that at the time. I had to add this prompt because it had Eren as the tea expert and Levi "more of a coffee drinker." Nope!) - have them order tea seated at a table with a waiter coming up. (It had them order at a counter, yet this is a posh restaurant? Really?) - add more dialogue (it had NONE)

- After the tea and snacks are finished, they leave the tea shop, walk to a park, and kiss by the river. (I had to add a prompt for an ending because it ended weirdly.)

Links

• Created Using Generative AI tag at AO3
  • AI-Generated Text

AI as a Translation Tool

Al can be used to translate, to assist in the translation or to proofread existing translations of fanworks. Some of them, like ChatGPT, will refuse to translate or proofread sexual content, even mild scenes, scenes that according to the algorithm are labeled as too violent or gritty or anything else that may go against the terms of service. There are internet tutorials to jailbreak ChatGPT and make it write texts that go against the TOS, but may stop working after a patch. Sometimes, the AI model will take creative licenses to fill in gaps and try to make sense of sentences with slang, or will remove sentences whose meaning it fails to convey or go against the TOS. Still, users consider it to be more accurate than any other translation tool available.

• Devils roll the dice (Angels roll their eyes) by LestappenForever (AI used for short English-to-French translations)
• Beyond Imagination by nakurumok (AI used to translate whole work from Chinese to English)
• spices by Tomoechka (AI used to translate whole work from Russian to English)

Music covers

AI tools are also used to produce music. AI covers typically replace the original voice with the voice of someone else, without changing the accent of the original singer or the style of the song. For example, in a cover of "Creep" by Radiohead sung by AI-generated vocals of Kurt Cobain, the voice would retain Thom Yorke's British accent, while the music would remain unchanged from the original song and not incorporate Nirvana's style. Some people manually create musical arrangements for their AI-generated vocals to better match the style of the band, as it was done in thie "Some Might Say" by Oasis AI cover sung in Beatles' style. A popular choice for AI covers is Frank Sinatra, with one notable example being an AI cover of Dragon Ball's "Cha-La Head-Cha-La" sung in Sinatra's usual style. Additionally, people create music covers using voice banks of politicians, fictional characters, or celebrities, such as Donald Trump or Peter Griffin from Family Guy, like this AI cover of "Somebody I used to know" by Gotye sung with AI generated voices of Homer and Marge Simpson.

K-Pop

In K-Pop fandom, AI covers are becoming increasingly more common, with practically every song receiving an plethora of AI-generated versions. Before the use of AI, a common trend on YouTube around the late 2010s was "How Would [Group] sing [Song]?". These were essentially "What If" scenarios, imagining a selected K-Pop group preforming a different group's song. In the 2020s, these types of videos have evolved to now include AI-generated vocals, often cloning the original artist, and sharing models across Twitter. For example:

• How would STRAY KIDS sing BITE ME by ENHYPEN
• NCT Dream "Midas Touch" - AI-generated models of a boy-group (NCT Dream) singing a girl group song (by Kiss of Life).
• making ‘Get up’ by Newjeans a full song with AI

Many artists are aware of the trend. Jungkook from BTS said they lacked sincerity. EXO member Baekhyun, suggested AI covers cannot compare to the originals. Bang Chan from Stray Kids called an AI cover of himself preforming "Seven" as "scary but cool". NCT Dream members dislike the covers.

Fanworks being used to train generative AIs

On December 1, 2022, Reddit user kaftherasu made a post on r/AO3 entitled "Sudowrites scraping and mining AO3 for it's writing AI" that started a 500-comment discussion. kaftherasu described some prompts given to an AI writing tool called Sudowrite and argued that Sudowrite's output strongly suggested its underlying dataset involved material scraped from AO3 (and/or other fanfiction sites); for example, when given the prompt "Steve had to admit that he had some reservations about how the New Century handled the social balance between alphas and omegas", Sudowrite's response indicated that it was familiar with Omegaverse tropes and that "Steve" was Steve Rogers (it added characters called Tony and Bucky). Other subsequent outputs featured members of BTS and references to Harry Potter. kaftherasu noted that as a result of the outputs, they had contacted the OTW to notify them of the suspected scraping. This incident specifically also inspired the event knotinmyname, where writers wrote omegaverse as a form of protest against AI.

The OTW later revealed in a statement on its stance on AI that code had been put in place in December 2022 to exclude AO3 from Common Crawl, a popular dataset to train AI language models such as ChatGPT and Sudowrite. However, it noted that works posted to AO3 before this change would already be part of the dataset.

On April 19, 2023, The Washington Post published an article entitled, 'Inside the secret list of websites that make AI like ChatGPT sound smart' that analysed Google's C4 dataset, "a massive snapshot of the contents of 15 million websites that have been used to instruct some high-profile English-language AIs, called large language models, including Google’s T5 and Facebook’s LLaMA." The article embeds a tool that allows readers to search the C4 dataset for specific websites and discover how prominently a website featured in the dataset. A search for Archiveofourown.org reveals that the site ranks 516th in the dataset (with 8.2 million tokens, or bits of text, from the site in the dataset), while Wattpad.com ranks 287th (12 million tokens). Fanfiction.net is not present, possibly due to the site disallowing web crawlers.

Numerous other fannish websites feature in the dataset, including (but not limited to):

• fanfiction.trekipedia.com (430,000 tokens)
• fanfictiony.com (150,000 tokens)
• fanfiction-library.com (89,000 tokens)
• fanfictionfestival.com (26,000 tokens)
• fanfictionfrenzy.com (23,000 tokens)

At the end of February 2024, leaked documents showed that Automattic (Tumblr's operator) was negotiating deals with OpenAI and Midjourney that would allow them to train AI models with user content on Tumblr and Wordpress. In response, Automattic announced that by default it blocked "major AI platform crawlers" on Tumblr and Wordpress.com, while acknowledging that it was "working directly with select AI companies as long as their plans align with what our community cares about: attribution, opt-outs, and control." Tumblr allows users to disable third-party sharing mentioning that doing so also prevents AI models from using content.

Further Information/Meta

Many fandom writers, podcasters, and Youtubers have broached the topic of fandom and AI in different ways including:

• Elizabeth Minkel wrote 'Sexy AI Chatbots are Creating Thorny Issues for Fandoms'
• Fansplaining (podcast) has dedicated two episodes to-date on the topic of AI, including referencing Elizabeth's article above: Episode 187: Artificial Fandom Intelligence and Episode 201: Artificial Fandom Intelligence 2: Rise of the Grifters
• ColeyDoesThings made a video 'we need to talk about AI chatbots and fandom'
• Dear Fanfiction Authors podcast had the episode, 'Will AI Kill Fanfic Writing?'
• The mathematical and statistical podcast 'Carry the Two' (which sometimes applies those fields to specific media), looked at AI and the WGA/SAG-AFTRA Strike in, 'Caitlin Parrish and Allyson Ettinger on AI & the WGA Strike'
• Theft of Fanfiction Perpetrated by Cliff Weitzman, WordStream, Speechify (December 2024)
• The OTW's journal Transformative Works and Cultures has put a call out for articles about AI and fandom for a journal release in 2025.