---
id: two-stage-user-styled-translation/03-research-covers-each-stage-separately-and-the
title: 'Two-stage styled translation exists only in pieces: 3. Research covers each
  stage separately, and the joint is untested'
report: https://ai.hiina.space/reports/two-stage-user-styled-translation.md
section: 3
version: 1
updated: '2026-10-27'
author: Hiina
---

Index of everything on this site: https://ai.hiina.space/index.md

# Two-stage styled translation exists only in pieces: 3. Research covers each stage separately, and the joint is untested

- Section 3 of 12 of the report "Two-stage styled translation exists only in pieces" by Hiina (v1, updated 2026-10-27)
- Report overview and contents: https://ai.hiina.space/reports/two-stage-user-styled-translation.md
- Previous: https://ai.hiina.space/reports/two-stage-user-styled-translation/02-humans-ship-variants-only-where-the-choice.md
- Next: https://ai.hiina.space/reports/two-stage-user-styled-translation/04-live-tools-are-single-stage-translators-with.md

## 3. Research covers each stage separately, and the joint is untested

### Interlingua failed on coverage; LLM-written notes revive it with the source kept

Classic interlingua MT survived only in controlled domains. Grammatical Framework translates by parsing into an abstract syntax and generating from it. Its 2014 wide-coverage translator handled 11 languages, with 10k-60k dictionary entries per language and about 65k abstract lemmas. Its own documentation lists unknown words breaking parses, context-free disambiguation, idioms coming out wrong and fallback to chunk-by-chunk translation, and says "the performance is not yet generally on the level with the best of the competition" ([GF translation doc](https://www.grammaticalframework.org/lib/doc/translation.html), project doc). The KANT group's 1992 paper title argues against pure transfer, pure interlingua and pure statistics alike ([ACL Anthology](https://preview.aclanthology.org/setup/1992.tmi-1.20), bibliographic page only), and Ranta et al.'s 2020 paper frames GF's move from controlled languages toward robust pipelines ([CL journal listing](https://submissions.cljournal.org/index.php/cljournal/article/view/1554), listing only). The AMR experiment is the closest modern test of a meaning pivot. One Transformer generating from English AMR into 21 languages beat an English-only baseline on 18 of 21, but on gold AMR it only matched an AMR-to-English-to-MT cascade (Spanish 21.7 vs 21.6, Italian 19.8 vs 19.6, German 15.3 vs 15.7, the last favouring the cascade), and human semantic accuracy averaged about 2 on a 1-3 scale ([Fan and Gardent 2020](https://arxiv.org/pdf/2011.05443)). **A pivot representation loses roughly a third of the meaning when the source is withheld**; that is the strongest single argument for passing the Japanese to the renderer.

The LLM era revived the idea as prompting rather than as a pivot, and every version keeps the source. MAPS has the model write keywords, topics and demonstrations about the source, filters them with quality estimation, then translates with the source plus the knowledge; the authors report fewer hallucinations, ambiguity errors and awkward style ([MAPS](https://arxiv.org/abs/2305.04118), abstract only; published in TACL 2024 as [Exploring Human-Like Translation Strategy with LLMs](https://aclanthology.org/2024.tacl-1.13/)). TransAgents runs a simulated publishing house (editors, translator, localisation specialist, proofreader) over a shared guideline of glossary, summary, tone, style and audience ([arXiv 2405.11804](https://arxiv.org/html/2405.11804v1); [TACL 2025](https://aclanthology.org/2025.tacl-1.42/)). Its d-BLEU was the lowest of the compared systems (25.0 against 43.1-52.2), yet monolingual readers preferred it over GPT-4 55.6% of the time and over a human reference 52.1%, which the paper itself calls "marginally preferred". It cost about $2.08 per chapter against $168.48 for a human at ATA minimum rates. TEaR (translate, estimate, refine) found one refinement round best and named estimation as the bottleneck: weak estimators overestimate errors and hallucinate corrections. Its arXiv v3 reports that 46.2% of evaluators preferred TEaR over the initial translation on Japanese-to-English in a forced choice ([TEaR arXiv v3](https://arxiv.org/html/2402.16379v3)); the published Findings of NAACL 2025 version covers 8 directions in 5 languages with no Japanese and no human preference study ([NAACL Findings 2025](https://aclanthology.org/2025.findings-naacl.218.pdf)).

Gains from explicit intermediate reasoning depend on the model. On EN-to-FR context-aware translation, reasoning raised GPT-4o's anaphora accuracy from 0.90 to 0.97 but lowered GPT-4 from 0.94 to 0.88 and GPT-4-turbo from 0.91 to 0.84, and cost DeepSeek-R1 8B and 14B 0.0298 and 0.0647 COMET on the anaphora translation task ([Ataee and Popescu-Belis 2025](https://arxiv.org/html/2510.18077v1)). Interpretability work suggests LLMs sometimes route word translation through an intermediate concept, though direct co-occurrence mapping dominates for frequent pairs ([Semantic Pivots](https://arxiv.org/html/2505.16385v1), not citation-checked). The lesson for this design: the strong model should do the reasoning offline, once, and the weak model should receive conclusions, not a long chain to re-derive.

### Formality control works into Japanese; this search found nothing out of it

Japanese formality control is a mature task in the English-to-Japanese direction. Feely et al. tagged Japanese targets as informal, polite or formal so that one model produces multiple registers from the same English ([WAT 2019](https://aclanthology.org/D19-5203/), abstract only). CoCoA-MT made Japanese the hardest language in its benchmark: formal and informal references overlapped least (BLEU 74.6), two commercial systems produced inconsistent "other" formality 24.2% and 20.0% of the time, annotator agreement was the lowest (Krippendorff's alpha 0.82), and the Japanese formality classifier reached about 0.7 precision with recall of 0.43-0.54. Fine-tuning on 1,000 contrastive segments raised in-domain accuracy from 64.5% formal and 35.5% informal to 84.8% and 84.4%, and contrastive data beat 2,000 or 10,000 non-contrastive segments. In human evaluation on held-out domains, informal accuracy rose 67.5 points while formal fell 13.0 ([CoCoA-MT](https://ar5iv.labs.arxiv.org/html/2205.04022)). Amazon's IWSLT 2022 system reached 88.8% formal and 98.8% informal on the blind test ([2022.iwslt-1.32](https://aclanthology.org/2022.iwslt-1.32.pdf)). Prompting caught up: Marrese-Taylor, Wang and Matsuo found GPT-3 and ChatGPT reach matched accuracy of 0.86 and 0.83 (at coverage 0.25) on English-to-Japanese, against 0.76 for the supervised Transformer of the CoCoA-MT paper ([WMT 2023](https://aclanthology.org/2023.wmt-1.49.pdf)). That paper's "compliance" figures (GPT-3 100%, ChatGPT 98.9%, Llama2-chat 13B 55.4% zero-shot and 91.3% one-shot) measure the share of outputs containing a parseable translation. Llama2's low number came largely from refusals, because it flagged words as rude or harmful. That gap is a refusal and format failure, not weaker style-following, and it is a warning for local aligned models rendering villains.

The direction in question here is the reverse: deciding what Japanese register and honorifics become in English under a user preference. **This search found no paper on Japanese-to-English honorific or keigo rendering control.** Two adjacent results transfer anyway. Multi-attribute prompting interferes: controlling one entity's gender lowered accuracy on another, often by around 10 points on WinoMT and by up to 24.2 points on the multi-entity GATE set, although single-entity accuracy (ChatGPT 94.4-96.5%) beat fine-tuned baselines near 87% ([Lee et al., preprint](https://arxiv.org/pdf/2407.15154)). Standard metrics also miss register: on JP-TL-Bench, a 350M MT model nearly matches GPT-4o on COMET (0.8889 vs 0.9093), but a rubric judge rated only 39.5% of its outputs perfect against 64% for GPT-4o. The benchmark's judge, gemini-2.5-flash, is also one of its anchor models, and the authors flag the self-preference risk ([JP-TL-Bench](https://arxiv.org/pdf/2601.00223)).

### Context helps in small doses, and puns mostly fail

For Japanese media, context helps in human judgement but saturates quickly. Scene-level NMT on manga scored 2.98 against 2.76 for sentence-level in human rating while BLEU fell from 14.11 to 12.65, and visual features added nothing significant ([Hinami et al., AAAI 2021](https://arxiv.org/pdf/2012.14271)). With GPT-4 Turbo on OpenMantra, page-level translation with the page image beat line-level; the best variant reached chrF 36.8 against 34.2 for Google Translate. The page-plus-image variant (chrF 36.6) scored MQM -1.98 against -1.31 for the official translation and -4.25 for Google, but had 160 critical errors against 107 for the official translation (Google: 272). Three-page and whole-volume context reduced quality, and a rolling chain-of-density story summary was the best long-context method yet still below page-level ([Lippmann et al., COLING 2025](https://aclanthology.org/2025.coling-main.232.pdf)). A Japanese study found LLaVA-generated image descriptions had minimal or negative effect on GPT-3.5 manga translation ([NLP2024 P7-13](https://www.anlp.jp/proceedings/annual_meeting/2024/pdf_dir/P7-13.pdf), not citation-checked). Generic description does not help; discourse-relevant facts might.

Smaller translation LLMs also fail to use raw context. Across nine models (Tower, ALMA, Llama-2, EuroLLM-9B), EN-to-DE pronoun accuracy with gold context was 26.1-35.5% against 33.3% chance, around or below chance, versus 56.2% for a concatenation-trained encoder-decoder. Perturbed context cost 5-10 points of pronoun accuracy while COMET barely moved ([Mohammed and Niculae 2024](https://arxiv.org/pdf/2410.14391v2)). That supports converting implicit context (who is "you" here) into explicit fields a small model can copy, rather than ten lines of history it will ignore.

Wordplay is where render-time generation is weakest. In the CLEF 2026 JOKER English-to-French pun task, 43% of submitted translations contained wordplay and about 27% were fully successful. The top system reached 79-82% wordplay rate but only 48-50% meaning preservation, while the most faithful system kept 77-78% meaning at about 29% wordplay. BLEU correlated at -0.08 with the number of successful translations ([JOKER 2026 overview](https://ceur-ws.org/Vol-4283/paper303.pdf)). "Literal versus localised puns" is therefore a real trade-off on two axes, and a small renderer will not recreate puns on the fly: the offline pass has to record the mechanism and propose candidate renderings for the renderer to choose between.

Style transfer research gives the same warning for restyling generally. On a content-preserving formality transfer test derived from task dialogue, the best system's joint score (style accuracy times meaning, slot preservation and fluency) was 0.46, and an unconstrained T5 kept content at 0.87 but style at 0.54 ([Don't lose the message while paraphrasing](https://ar5iv.labs.arxiv.org/html/2308.09055)). Combined with the IWSLT result that rule-based editing beat a learned post-editor, the design rule is to move every closed-class transform into code and reserve the model for open-ended voice.

---

Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space