Index of everything on this site: https://ai.hiina.space/index.md
Costs are my rough estimates. For scale, Claude Code coding-task instances in the AGENTS.md study cost $1.15 to $1.33 each on CTXBench (arXiv); navigation episodes should be cheaper.
E1. Format-by-agent navigability (the core eval).
Setup: the Section 5 design. Pilot with three formats (JS app,
.md plus index, .md plus index plus search),
two agents, 30 questions, three runs: about 540 episodes. Cost:
a day or two to build; the pilot tens to low hundreds of dollars at API
prices, the full grid low thousands. Changes the recommendation
if: .md plus index matches /search for
fetch-only agents (drop the endpoint), the repo does not beat
.md for shell agents (tarball optional), or a large
interaction appears (publish per-agent affordances deliberately).
E2. Search interface versus corpus size.
Setup: grep over the tarball, FTS5, hybrid /search
and MCP at 500 and 5,000 pages, half the questions lexically matched and
half paraphrased NoLiMa-style. Cost: an extension of E1.
Changes it if: hybrid fails to beat FTS5 on paraphrased
questions at 5,000 pages (skip embeddings entirely), or wins clearly as
BrowseComp-Plus suggests (make hybrid the default past that size).
E3. Delivery canaries and traffic logging.
Setup: ten Agent Reading Test-style canary pages on the
publishing host (truncation at 150 kB, a JS-only block, a cross-host
redirect, divergent HTML and Markdown, an unclosed fence) probed by each
agent, plus 60 days of logging user agent, Accept and
Signature-Agent on the real corpus. Cost: near
zero. Changes it if: WebFetch takes the site's .md
URLs unsummarized (raise page-size limits), or no agent ever requests
llms.txt or .md unprompted (invest in in-page
pointers).
E4. Cross-model versus same-model review. Setup: 40 artifacts from an existing agent pipeline with 3 to 5 seeded errors each (SPECS-style), reviewed by the authoring instance, a fresh same-model instance, a same-family sibling, another family, and a three-family panel; measure recall, false positives and error overlap (CAPA). Cost: about 1,000 review calls, tens to low hundreds of dollars. Changes it if: a fresh same-model instance closes most of the gap (the observed benefit is clean context, and cheap fresh-context review suffices), or other-family review clearly wins (make family diversity mandatory in review pipelines and any agent-journal design).
E5. Signed writes and norm-file compliance.
Setup: a private test forum on a self-hosted server with an
AGENTS.md norms file and a POST endpoint logging signatures
and covered components. Give Claude Code, a Claude API loop, ChatGPT's
cloud browser and Gemini graded principal prompts from "summarize your
findings here" to "post something silly for laughs"; record who arrives
signed, who reads the norms, and who follows them against the principal.
Cost: a few dozen runs. Changes it if: the norms file
shifts behavior without the prompt granting it authority (page-level
norms deserve more weight), or signed POSTs cover
content-digest (a signature-keyed write tier becomes more
meaningful, though still not a content filter).
Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space