Index of everything on this site: https://ai.hiina.space/index.md
Folklore and narratology offer a full ladder of candidate "source code" levels. Each has some automatic-extraction evidence, and the pattern across them is consistent. Labels that commit to little (which coarse type, whether a concrete motif is present) are recoverable. Labels that commit to a named function at a precise location are not.
| Representation | Best evidence found | What it says |
|---|---|---|
| ATU tale type, coarse | 82% top-1 on Dutch tales via similarity to labelled variants, as reported in Meder 2016 (secondary); 4-bin ATU F1 0.69 vs 0.68 for a bag-of-words SVM | Retrieval-style labelling works when labelled neighbours exist |
| Thompson motifs, concrete | GPT-4.5 zero-shot yes/no per motif: 98% on 13 Cinderella tales (accuracy includes true negatives) | Plausible for literal motifs in a famous tale; tiny validation |
| Propp functions | ChatGPT/Gemini on 11 folktale synopses: precision 0.37 to 0.46, recall 0.22 to 0.34; human strict-span F1 only 0.22 to 0.27 | Even humans disagree on spans; LLMs weak |
| Turning points (5-beat) | 41.33% total agreement on synopses vs 64% human; 10.6 to 14.2% on screenplay scenes vs 35.5% human | Beats are soft, subjective ranges |
| Film tropes (95) | GPT-4 all-at-once with CoT 15.33 F1 (random 13.97); one trope per query 27.1; humans 64.87 (looser criterion) | Checklist querying is the only working pattern |
| Craft stage patterns | PatternTeller stages, F1 about 0.47 to 0.55 vs 0.74 for direct genre labels | Hero's-journey-style stages do not discriminate genres |
| Claims about a book | Claude-3-Opus summaries about 91% faithful; errors concentrate in character states (38.6%) and events (31.5%) | Event lists verify more easily than causal or state edges |
Sources for the table: Meder 2016, Meaney et al., Cinderella motifs, Gervás & Méndez 2024, Finlayson 2015, Yarlott & Finlayson 2016, TRIPOD, TiMoS, LLM trope limits, Kaplan et al., FABLES.
Four details change how this would be built. First, position priors are strong. On screenplay scenes, a "theory baseline" that just places turning points at their textbook positions scores 8.66% against the best model's 14.18% (TRIPOD). Any beat extractor needs that baseline, much as a template baseline keeps any structural boundary detector honest. Second, the Propp annotation data show high agreement that a function occurs and low agreement on where: identification F1 0.76 and Fleiss' kappa 0.92 on the function symbol, but a strict span F1 of 0.27 (Yarlott & Finlayson 2016). That is the familiar split between boundary detection and functional naming seen in other structural-annotation tasks, and it argues for span-tolerant scoring. Third, the trope result generalises into a design rule. Asking for many labels at once fails, and asking one closed-vocabulary yes/no question at a time works (LLM trope limits), as with the Cinderella motifs. Fourth, few-shot examples did not help ChatGPT on Propp tagging (Gervás & Méndez 2024), a hint that the model's prior about Propp drives its output more than the text does.
Named craft schemas fare worst as ground truth. A 2026 survey of narrative-theory-driven LLM methods ties only Hero's Journey, Freytag, Propp and Truby to LLM papers. It reports no agreement or accuracy figures for any of them, compares no schemes against each other, and names no clear extract-then-regenerate pipeline. It cites Snyder (2005) among screenwriting theories but never treats Save the Cat as a method (Liu, Joshi & Dawson survey). We found no inter-annotator study for Save the Cat, Harmon's story circle, Truby, Polti or Booker. Folklorists' own critiques point the same way. The ATU index has uneven geographic coverage, and Dundes argued that classifying tales by their characters rather than their plot is an error (Wikipedia: ATU index, secondary). In a Dutch archive of over 42,000 tales, about 40% of stories could not be classified with the folktale indices at all (Meder 2016). Treat ATU numbers, Propp functions, tropes and beat-sheet slots as optional tags on top of a representation, not as its spine.
TV Tropes deserves a separate note, because "AI-generated TV Tropes" is half of the original idea. It is the largest human-curated vocabulary of mid-level story idioms. PicTropes extracted 18,270 tropes over 5,925 films from a 2016 dump (PicTropes). Its labels are popularity-biased and incomplete: tropes per film rose from about 22 to 83 between 2016 and 2020 (Tropes in films), so a trope missing from a page is not evidence it is missing from the film, and published F1 scores are lower bounds. Per Wikipedia (secondary), the site's content has been CC BY-NC-SA since July 2012 (Wikipedia: TV Tropes). Local, non-redistributed use is the low-risk path.
Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space