Markdown version: https://ai.hiina.space/reports/story-decompiling-and-remixing/04-plan-conditioned-decoders-drop-a-quarter-of.md ยท index

Index of everything on this site: https://ai.hiina.space/index.md

Story decompilers exist, but nobody closes the loop: 4. Plan-conditioned decoders drop a quarter of the plan and invent the rest

4. Plan-conditioned decoders drop a quarter of the plan and invent the rest

The decoder half is the best-developed part of the literature. Every system since 2018 conditions on some intermediate plan, and the representations form a ladder of compression. At the top are one keyword per sentence (Plan-and-Write), then semantic-role event tuples (Aristotelian rescoring, Ammanabrolu's event 5-tuples), then premise, characters and numbered outline (Re3), then hierarchical outline trees with per-node characters and settings (DOC), then logline to characters to beats to locations to dialogue (Dramatron), and finally role-specialised planning agents writing to a shared scratchpad (Agents' Room). Human raters usually prefer the planned output, but the gains are modest and rarely isolated with a fixed backbone (Re3; DOC; Dramatron; Agents' Room).

The numbers that matter for an autoencoder are adherence numbers, and they are sobering. Plan-and-Write realised only 75 to 78% of its storyline keywords, and plan fidelity carried only 19.7% of the weight in human preference, against 41.4% for coherence (Plan-and-Write). Goldfarb-Tarrant et al.'s story model used 72.0 to 75.5% of plot words and verbs, with the rescored system at the low end (Aristotelian rescoring). DOC's controller raised the share of passages faithful to their outline leaf from 37.8% to 58.5%. The system rarely strayed from the top-level plan but often missed lower-level details (DOC). Re3 reports an outline-mandated affair that simply never happens (Re3). Agents' Room, the strongest modern system, never measures whether stories follow their plans at all. Its LLM judge correlates with humans at rho 0.62 across systems but only 0.41 per item (Agents' Room). Consistency errors grow with length and cluster mid-story (ConStory-Bench). DOME's ablation shows what a running fact store buys: removing its temporal knowledge-graph memory raised the conflict rate from 0.56 to 4.52 (DOME).

The gap-filling that makes modern decoders fluent is exactly what a round-trip metric will register as divergence. Ammanabrolu et al. call eventification "a lossy process" and needed a five-model fallback cascade to turn tuples back into sentences (Story Realization). The clearest empirical preview of the bottleneck problem comes from screenwriting. When Lei et al. derived training inputs by decomposing real screenplays ("reverse synthesis"), props and minor characters in the target were missing from the input outline. Training only on such pairs scored 7.14 on their 12-point expert rubric, against 8.06 when mixed with forward-synthesised data (Beyond Direct Generation). A plan cannot carry everything, so the decoder invents the rest, and the metric has to separate harmless invention from lost plot.

Granularity evidence is thin. No paper compares logline, beat sheet, scene list and event graph on a common benchmark. The consistent pattern across DOC and DOME is two-level: a stable coarse spine plus per-node expansions with explicit characters and settings that adapt as text is written. This matches the plan-then-realise result that a companion report on two-stage user-styled translation also leans on. On WebNLG, a separate planner cut omissions from 41 to 6 at equal BLEU, but the realiser covered only 66.7% of plan entities on unseen data, against 98.9% on seen data (Moryossef et al. 2019). Story records unlike anything the decoder was trained on will drift the same way.


Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space