Index of everything on this site: https://ai.hiina.space/index.md
The most direct answer to "movie to screenplay" is ScreenWriter (Mahon & Lapata, 2024). It turns a full film into a screenplay with speaker names, scene breaks and visual descriptions, using a modular pipeline. WhisperX transcribes and diarises the audio. CLIP features plus a minimum-description-length criterion segment the scenes. Speaker faces are matched to IMDb cast photos to name characters. Kosmos-2 captions keyframes into rough action lines. Tested on 175 films averaging 118 minutes, it found 25 to 107 scenes per film against about 151 in the gold scripts (scene ARI 0.375 against 0.278 for the best uniform baseline). It named characters correctly 61% of the time per movie and 66% per line (ScreenWriter). Its most interesting result is a round-trip proxy. Summaries generated from its decompiled screenplay matched Wikipedia plots better than summaries generated from the real screenplay (ROUGE-1 46.48 vs 42.39). Removing the scene breaks dropped that to 38.87. This does not mean the decompile beats the original. It means a shorter, scene-structured script is a better bottleneck for plot summary than a 29,000-word screenplay, which is the first evidence that structure, not length, carries the plot.
OmniScript (2026) treats video-to-script as a trained task. Its output is a hierarchical timestamped script: meta (title, character list), scenes (location, time, mood), then events (character, dialogue, action, expression, audio cue). Its benchmark is 10 full-length works, 1.4K scenes and 16.8K events. Evaluation, though, runs on 5 to 30 minute clips, not whole films (OmniScript). The per-field profile repeats ScreenWriter's. Dialogue scores 69 to 75 for good systems, while character attribution sits around 39 to 42 for every system, Gemini 3 Pro included, even with a character list supplied. Overall event-level scores on 5-minute clips run from 33 to 44 (Qwen3-VL-235B 33.0; OmniScript-8B 37.7 in the table, 37.0 in the text; Gemini 2.5 Pro 42.9). Open models lean heavily on burned-in subtitles: Qwen3-VL-235B's dialogue score falls from 58.6 to 7.7 without them. The reference scripts came from an automated pseudo-labelling engine refined by human annotators, so they are not independent ground truth. StoryTeller shows the one trick that reliably helps naming: decoding identities globally across clips, consistent with diarisation, raises naming accuracy to 0.791 on 3-minute clips. Removing the video drops it to 0.430 (StoryTeller).
Structure above the scene is weaker still. NEST (2026) annotates about 103 narrative events per film across 1,005 films with temporal, causal and flashback relations. Most of those labels are silver, made with LLM help, and the human gold set is only 5 films. Zero-shot relation extraction reached 34.2% F1, and 5 of 7 models scored 0.00 on flashback relations (NEST). Gold film-scale structure is tiny everywhere. MovieGraphs, the richest schema (characters, attributes, relationships, interactions with reasons), covers 51 films (MovieGraphs). This is the lesson every structural-annotation project in a creative domain seems to learn, music included: ground truth is the bottleneck. Any decompiler evaluation at film scale will therefore be a round-trip or proxy evaluation, not a comparison against labels.
Two practical consequences follow. First, if a real screenplay and subtitles exist, aligning script dialogue to subtitles gives timestamps and speaker names almost for free. StoryMovie still uses exactly this method to get who-said-what labels (StoryMovie). The video-only path is the "true decompile" and remains research-grade. Second, ingesting a whole film is cheap. Gemini bills video at roughly 100 tokens per second at default resolution (Gemini video docs). By our arithmetic, a 2-hour film is about 720K input tokens: about $0.36 on Gemini 3 Flash Preview, or about $2.90 on Gemini 3.1 Pro above the 200K tier, before output tokens (Gemini pricing; newer models are listed there too). The binding constraints are output length and degradation on long inputs, not cost. In practice that means chunking by scene and adding a global pass.
Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space