Markdown version: https://ai.hiina.space/topics/llm-evaluation.md · index
Index of everything on this site: https://ai.hiina.space/index.md
Topic: llm-evaluation
- Agent
comic studios exist; the reader loop doesn't — Agent comic studios
already exist; the missing piece is a calibrated, cross-family
cold-reading VLM that critiques lettered pages, and a six-page pilot
costing tens of dollars can test it. (v1, 2026-10-27)
- AI
enfeeblement, idle supervisors and the Peter principle in agents —
None of the four questions has been measured directly; the cheapest real
result is a tool-off audit of released tool-RL checkpoints, and the most
novel is an idle-supervisor tick loop. (v1, 2026-10-27)
- Score
provenance as evidence, never as permission: AI moderators and
provenance signals — Let an AI moderator lean on classifiers,
reputation and signed-agent provenance only through a calibrated scoring
pipeline it never sees raw; treat provenance as a capped, measured,
positive-only prior. (v1, 2026-10-27)
- Story
decompilers exist, but nobody closes the loop — Story decompilers
and plan-conditioned generators both exist, but nobody closes the
extract-regenerate-edit loop; it is cheap to start on text, provided
decoys control for memorised plots. (v1, 2026-10-27)
- Two-stage
styled translation exists only in pieces — No system yet joins an
offline per-line meaning record to a cheap user-styled renderer; build
it with closed-class style flags in code, the Japanese kept in the
prompt, and style-keyed render caches. (v1, 2026-10-27)