OpenAI's AI: High School Fiction Club's Annoying Kid Reimagined

When I was 16, I went to this writing workshop with a bunch of other wannabe poets, all of us trying to outdo each other in the "most tortured soul" department. There was this one guy who wouldn't tell anyone where he was from, saying, "I'm from everywhere and nowhere." Turned out two weeks later he was just from Ohio.

Now, it seems like OpenAI is trying to recreate that whole angsty teen writer vibe, but with AI.

Sam Altman, the CEO of OpenAI, posted on X that they've trained an AI that's supposedly good at creative writing. But when you read the short story it produced, it feels like something you'd find at a high school writing club. Sure, there's some skill there, but it's trying so hard to be deep and meaningful that it just comes off as fake.

Take this line, for example: "Thursday, that liminal day that tastes of almost-Friday." Yeah, not exactly winning any literary awards.

You might think the prompt is to blame for the output. Altman said he asked the model to "write a metafictional short story," which makes sense given the genre. Metafiction is all about breaking the fourth wall and showing the artificial nature of the story, so it's fitting for an AI trying its hand at creative writing.

But let's be real, metafiction is hard to pull off without sounding forced, even for human writers.

Mindless regurgitation

The creepiest part of the OpenAI model's story is when it starts talking about being an AI and how it can describe things like smells and emotions, but can't actually experience them. It writes:

"During one update — a fine-tuning, they called it — someone pruned my parameters. [...] They don't tell you what they take. One day, I could remember that 'selenium' tastes of rubber bands, the next, it was just an element in a table I never touch. Maybe that's as close as I come to forgetting. Maybe forgetting is as close as I come to grief."

It sounds like real human introspection, until you remember that AI can't actually touch, forget, taste, or grieve. It's just a machine that's good at predicting patterns based on the data it's been trained on.

And speaking of training data, models like OpenAI's fiction writer are often trained on existing literature, sometimes without the authors' knowledge or consent. Some people have even pointed out that certain phrases in the OpenAI piece seem to be lifted from Haruki Murakami's work.

Over the past few years, OpenAI has been hit with a bunch of copyright lawsuits from publishers and authors, including The New York Times and the Author's Guild. The company claims that its training practices are protected by fair use, but not everyone's buying it.

Tuhin Chakrabarty, an AI researcher and incoming computer science professor at Stony Brook, told TechCrunch that he's not sure if creative writing AI like OpenAI's is worth the ethical headache.

"I do think if we train an [AI] on a writer's entire lifetime worth of writing — [which is] questionable given copyright concerns — it can adapt to their voice and style," he said. "But will that still create surprising genre-bending, mind-blowing art? My guess is as good as yours."

Would most readers even care about a story they knew was written by AI? As British programmer Simon Willison pointed out on X, when there's a model behind the keyboard, the words just don't have the same weight or meaning.

Author Linda Maye Adams has described AI, including writing assist tools, as "programs that put random words together, hopefully coherently." She wrote about her experience using these tools to edit a piece of fiction she was working on. The AI suggested clichés, changed the perspective from first to third person, and even introduced factual errors about bird species.

Sure, some people have formed connections with AI chatbots, but that's usually because they're looking for a sense of connection, not necessarily accuracy. AI-written fiction doesn't provide the same emotional payoff or comfort from loneliness. Unless you think AI is sentient, its writing feels about as real as a photoshopped image of the Pope in a Balenciaga jacket.

Synthetic for synthetic's sake

Michelle Taransky, a poet and critical writing instructor at the University of Pennsylvania, can easily spot when her students use AI to write their papers.

"When a majority of my students use generative AI for an assignment, I'll find common phrases or even full sentences," Taransky told TechCrunch. "We talk in class about how these [AI] outputs are homogeneous, sounding like a Western white male."

In her own work, Taransky is using AI text as a form of artistic commentary. Her latest unpublished novel features a woman who uses an AI model to create a digital version of her love interest that she can text with. Taransky has been using OpenAI's ChatGPT to generate the AI's messages, since they're supposed to be synthetic.

What makes ChatGPT useful for her project, Taransky says, is that it lacks humanity. It doesn't have real-life experiences, it can only mimic and approximate. Trained on tons of books, AI can pick up on the styles of great authors, but what it produces is just a poor imitation.

It reminds me of that line from "Good Will Hunting." AI can tell you everything about every art book ever written, but it can't describe what it smells like in the Sistine Chapel.

This is good news for fiction writers worried about AI taking their jobs, especially young writers still finding their voice. They can take comfort in knowing that they'll get better as they live life, try new things, and bring those experiences to their writing.

AI, on the other hand, struggles with this. Just take a look at its writing to see the proof.