Index of everything on this site: https://ai.hiina.space/index.md
The edit half has one mature benchmark, and its lessons transfer more than its scale does. TimeTravel gives a five-sentence story and a counterfactual premise, and asks for an ending rewritten with minimal change: 29,849 human rewrites (TimeTravel). Its history is a warning about metrics. A supervised model beat the human references on BLEU-4 (76.63 against 65.12) simply by copying the original ending. BLEU and ROUGE correlated negatively with human judgements of counterfactual adherence (TimeTravel full text). The field moved to two-sided scores: HMEAN of overlap and entailment, which correlated best with humans at r = 0.50 (EDUCAT), and DTO's delta score, which asks whether the output is closer to the edited reference than to the original (DTO). DTO also shows that LLMs over-edit: GPT-4o zero-shot preserved less of the text that should stay unchanged (ROUGE-L 46.9) than a fine-tuned baseline (60.9). For a story remix, that becomes three checks: the requested change is present, the plot is preserved outside the edit's scope, and nothing else drifted.
Whole-story transposition ("X but Y") has no benchmark we could find, and the closest results predict trouble. StoryTrans, a discourse-level style-transfer system, states outright that "content preservation and style transfer are adversarial" (StoryTrans). FlawedFictions found that LLM-written contemporary adaptations of stories raised the plot-hole rate from 0.14 to 0.27 (Claude 3.5 Haiku) and to 0.53 (GPT-4o-mini). The best detector, Claude 3.5 Sonnet, matched human accuracy at 0.76 on short stories but fell to chance on 2,700-word tales (FlawedFictions). Gender-swap work in narrative is almost all bias auditing. A FairytaleQA study swapped pronouns and gendered nouns by rule and by LLM, but measured a downstream QA model, not story coherence (FairytaleQA CDA). A Portuguese-literature study found LLM rewriting reproduced, and sometimes amplified, gendered role stereotypes (JBCS 2025, abstract only). A "gender-swap, keep plot" instruction can therefore silently reassign roles. In Hamlet, succession, marriage and the Ophelia plot all depend on gender, and no published work tests whether such dependencies propagate. StoryAnalogy gives a candidate invariant for transposition. Its analogy score is high when relational structure matches and entities differ. ChatGPT identified such analogies about 30% of the time, against 85.7% for humans (StoryAnalogy).
Two findings argue for putting the checking outside the writer. Wang & Kreminski found frontier models handle causal soundness in small narrative-planning domains, but character intentionality is much harder: o1 and Claude 3.7 scored 14 and 15 of 30 on the Aladdin domain. External calibration lifted gpt-4o from 13 to 29 of 30 on causal soundness (Wang & Kreminski). That is the classic narrative-planning line, IPOCL treating a story as a causally and intentionally valid plan (Riedl & Young), returning as a verifier. Long-horizon consistency stays poor without such structure: in interactive narratives, the best model's survival rate after 20 turns was 42%, and fact conflicts appeared in 40 to 68% of runs across models (NCP-Bench, abstract only). Compound requests will also degrade by count. The FollowBench numbers that also matter for styled translation apply directly: GPT-4's hard-constraint satisfaction fell from 84.7% at one constraint to 61.9% at five (FollowBench). "Hamlet but Lovecraftian and gender-swapped" is at least three stacked constraints on top of plot fidelity.
Folklore and game decompilation both suggest the same layering for
the representation. Folklorists treat the tale type as
a self-sufficient plot and the motif as the smallest
recombinable element (Harvard
tale-type guide). Von Sydow's ecotype describes tales adapting their
surface to local conditions while stabilising within cultural districts
(JEF
2019, abstract level). "Shakespeare recast in another setting" is
the same move: hold the type, swap the local layer. The Super Mario 64
decompilation is remixable because of three properties. It can be
checked against the original ROM's hash, with NON_MATCHING
builds that are "functionally equivalent" but not byte-identical. It
separates code from data such as levels, behaviours and text,
re-extracting the assets from the user's own ROM. And it exposes named,
typed entities (n64decomp/sm64). N64Recomp
supports mods by replacing functions (N64Recomp). Cross-game
ports mostly wrap or reimplement the guest rather than translating it.
Mario in Elden Ring reportedly uses libsm64 (unwire.hk,
secondary), and IW4L is a from-scratch Rust rewrite of Modern Warfare 2
(Held
Games roundup, secondary). The analogy holds in one direction and
breaks in another. Stories have no executable semantics, so there is
never a "matching" build. The best available is "NON_MATCHING but
functionally equivalent", judged by lossy checks. What transfers is the
architecture: one source of truth with checkable claims, persistent
state separated from events, and edits at named interfaces rather than
arithmetic on a global latent.
Re-cut trailers mark the boundary of that interface. Robert Ryang's family-comedy cut of The Shining left the footage untouched and changed only music, voice-over and shot order (Wikipedia: Re-cut trailer, secondary). Much of genre lives in framing variables. A real Lovecraftian Hamlet changes motivations and causal mechanics, though, which re-framing cannot do. Industry already runs this layering forward. A series bible holds persistent facts (names, a favourite bar, a food allergy) for continuity (Wikipedia: Bible, secondary). Beat boards hold movable event cards colour-coded by storyline (Final Draft). Dramatron's logline, characters, beats, locations and dialogue ladder mirrors this (Dramatron).
The schema this evidence supports has four layers. The first is a logline and theme. The second is a bible of entities, each with a narrative role (Proppian function) kept separate from identity attributes (name, gender, species), plus world rules and a setting layer. The third is ordered scene cards: participants, location, goal, outcome, state changes, and who knows what. The fourth holds optional soft tags (ATU type, Propp function, tropes, beat names, an emotional arc), each queried as a checklist. Edits then sort into three classes. Identity and surface swaps (names, pronouns, setting nouns) are closed-class and belong in code, the same conclusion the companion translation report reaches for honorifics. Setting swaps replace the world layer and need a propagation pass over the scene cards. Genre changes rewrite motivations and causal links, and are the only class that should go to an LLM planner plus an external consistency check. Emotional arcs fit in the fourth layer as a cheap, deterministic control signal, not as the latent. Reagan et al. themselves say the arc "does not give us direct information about the plot" (Reagan et al.).
Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space