TIME – Read More
Not only are people unable to tell the difference between stories written by humans or artificial intelligence, they actually prefer it when the stories are created by AI. And that’s especially true when they are told that the stories had human authors, according to a new study from Villanova University.
The study consisted of multiple experiments, all of which indicated a preference for AI. In the first, 1,682 participants were told to rate both the quality of six short stories they were given and how “engaging” they were. They were told, either correctly or incorrectly, who or what had authored each one.
Read More: Is AI Making Our Brains Weaker?
Not only were AI-generated stories found to be higher-quality (by 6%) and more engaging (by 8%), they were rated “even more highly,” researchers concluded, when participants believed that they were written by humans (by an additional 3%).
Deena Weisberg, the senior author of the study and an associate professor from Villanova’s Department of Psychological & Brain Sciences, said in a press release that the finding “reveals a bias towards narratives written by real people.”
That may be because “we assume creative writing requires uniquely human qualities, such as emotional understanding and lived experience,” she said, explaining that it shows how “public assumptions about AI’s capabilities are increasingly out of date.”
Furthermore, people were only able to identify AI-generated content roughly 40% and 52% of the time in two subsequent experiments, each with over 400 participants. People who said they were AI-literate had more success, while people who claimed to have a background in literature were less accurate in their guesses.
“Familiarity with AI systems appeared to help people recognize the patterns typical of AI-generated writing, such as em dashes and sentence structures such as ‘it’s not just X, it’s Y,’” Weisberg said. “That suggests that improving AI literacy would be one way to help people to navigate the new AI-enabled world that we’re living in.”
Cameron Jones, assistant professor of psychology at Stony Brook University tells TIME that the study’s findings are not necessarily surprising, considering the history of research going back years that shows people’s diminishing ability to tell between artificial intelligence and humans.
He also points to the trajectory of his own studies, which look at whether people can differentiate between them in conversation. In those scenarios, he says, “The person actually gets to quiz the model and ask follow-up questions.”
Even then, they had trouble distinguishing between human and generative content partners.
Jones says that this is because the large language models driving the content responses are trained to appeal to users.
“We basically get the model to generate little bits of text, and then we get people to read them, and they give a thumbs up if they like what the model’s saying and they give a thumbs down if they don’t like what the model’s saying,” he says.
It’s no surprise that people prefer the results, he explains, when “we’re optimizing these models’ outputs against our preferences.”
It can oversimplify output, but it leads to writing with higher overall appeal—whereas humans might write something truly exceptional that only caters to select tastes.
“Real humans are weird and idiosyncratic, and they’re very different from one another,” Jones says. “But models can kind of learn to be this kind of milquetoast everyman who appeals to everybody.”
While the findings align with rising exposure to artificial intelligence, along with algorithmic reward systems for broadly approachable content on platforms like TikTok, Weisberg does not think that the current digital landscape is to blame for the results. She says, “Technologies amplify existing tendencies, rather than creating them.”
Plus, she adds, the human-authored content might have simply challenged readers more.
“AI writing tends to be clearer, more direct and easier to process,” she said. And it’s reasonable for participants to find themselves more deeply engaged with content that offers greater predictability and less friction.
In other words, Weisberg says, “Difficult or subtle material requires more brainpower.”
