StoryScope Detects AI-Written Stories by Plot Structure, Not Style

A team of researchers from the University of Maryland and Google DeepMind has developed a tool called StoryScope that identifies whether a short story was written by a human or a language model by looking solely at plot structure, not sentence style. Accuracy tops 93 percent, even when the text has been stylistically smoothed beforehand to fool classic AI detectors.

How StoryScope works

The tool breaks down each story into ten narrative dimensions drawn from the NarraBench taxonomy: characters, social networks, events, plot structure, setting, temporal organization, how information is revealed, point of view, style markers, and overall structure. GPT-5.1 converted each story into a structured JSON template, from which as many as 304 features per story were extracted.

For testing, the researchers assembled a corpus of 10,272 writing prompts, each written once by a human and once by five different language models, yielding 61,608 stories of roughly 5,000 words each. From this, the classifier learned to distinguish human and machine prose without looking at word choice or sentence rhythm.

How AI prose differs

The biggest difference lies in how the narration handles the story's meaning. Narrators in AI-written texts explicitly name the story's theme or moral in 77 percent of cases, compared to 52 percent for human authors. Language models more rarely let readers arrive at meaning on their own, more often spelling it out directly.

A similar pattern shows up in dialogue. In AI stories, 59 percent of conversations between characters serve as philosophical debate about what the story means, versus 34 percent in human-written ones. Plots generated by models are also structurally simpler, only 21 percent contain subplots, while 43 percent of human-authored stories do, and AI stories tend to follow a more linear, cause-and-effect resolved arc.

The researchers also found that AI-generated stories cluster in a narrow region of narrative space, while human texts are far more varied. In other words, language models write plots in a handful of repeating ways, while humans write them in far more ways.

Each model's fingerprint

StoryScope can also identify which specific model wrote a given text, with over 68 percent accuracy across six sources. Each model has its own telltale pattern: Claude produces a flattened escalation of events, GPT overuses dream sequences, and Gemini favors external, physical description of characters over their inner lives.

In an additional test, the researchers deliberately smoothed the style of AI texts, stripping out typical stylistic tells that might give away their origin. The accuracy of the plot-structure-based classifier barely changed, showing that the detector doesn't rely on word choice or sentence rhythm but on the story's deeper architecture.

What this means for publishers and universities

The findings arrive as publishing houses, literary award committees, universities, and courts struggle with the inadequacy of existing style-based AI detectors, which are increasingly easy to fool with simple rewriting. A method that analyzes plot structure instead gives them a tool that's harder to defeat just by instructing a model to write more like a human.

For writers using generative AI in literary work, there's a practical takeaway: what matters isn't avoiding artificial-sounding sentences, but changing the plot's construction, adding subplots, moral ambiguity, and nonlinear chronology, the things that today naturally set human stories apart from machine-written ones.

The work was carried out jointly by the University of Maryland and Google DeepMind, showing that major AI labs are themselves investing in research into detecting content generated by their own models, alongside their work on developing them.

Sources: Tech Times (techtimes.com), StoryScope research paper on arXiv (arxiv.org).