---
id: agent-readers/09-experiment-plan
title: 'Agent readers and agent institutions: 9. Experiment plan'
report: https://ai.hiina.space/reports/agent-readers.md
section: 9
version: 1
updated: '2026-10-26'
author: Hiina
---

Index of everything on this site: https://ai.hiina.space/index.md

# Agent readers and agent institutions: 9. Experiment plan

- Section 9 of 11 of the report "Agent readers and agent institutions" by Hiina (v1, updated 2026-10-26)
- Report overview and contents: https://ai.hiina.space/reports/agent-readers.md
- Previous: https://ai.hiina.space/reports/agent-readers/08-recommendations-the-stack-to-publish-on-now.md
- Next: https://ai.hiina.space/reports/agent-readers/10-conclusion.md

## 9. Experiment plan

Costs are my rough estimates. For scale, Claude Code coding-task instances in the AGENTS.md study cost $1.15 to $1.33 each on CTXBench ([arXiv](https://arxiv.org/abs/2602.11988)); navigation episodes should be cheaper.

**E1. Format-by-agent navigability (the core eval).** *Setup:* the Section 5 design. Pilot with three formats (JS app, `.md` plus index, `.md` plus index plus search), two agents, 30 questions, three runs: about 540 episodes. *Cost:* a day or two to build; the pilot tens to low hundreds of dollars at API prices, the full grid low thousands. *Changes the recommendation if:* `.md` plus index matches `/search` for fetch-only agents (drop the endpoint), the repo does not beat `.md` for shell agents (tarball optional), or a large interaction appears (publish per-agent affordances deliberately).

**E2. Search interface versus corpus size.** *Setup:* grep over the tarball, FTS5, hybrid `/search` and MCP at 500 and 5,000 pages, half the questions lexically matched and half paraphrased NoLiMa-style. *Cost:* an extension of E1. *Changes it if:* hybrid fails to beat FTS5 on paraphrased questions at 5,000 pages (skip embeddings entirely), or wins clearly as BrowseComp-Plus suggests (make hybrid the default past that size).

**E3. Delivery canaries and traffic logging.** *Setup:* ten Agent Reading Test-style canary pages on the publishing host (truncation at 150 kB, a JS-only block, a cross-host redirect, divergent HTML and Markdown, an unclosed fence) probed by each agent, plus 60 days of logging user agent, Accept and `Signature-Agent` on the real corpus. *Cost:* near zero. *Changes it if:* WebFetch takes the site's `.md` URLs unsummarized (raise page-size limits), or no agent ever requests `llms.txt` or `.md` unprompted (invest in in-page pointers).

**E4. Cross-model versus same-model review.** *Setup:* 40 artifacts from an existing agent pipeline with 3 to 5 seeded errors each (SPECS-style), reviewed by the authoring instance, a fresh same-model instance, a same-family sibling, another family, and a three-family panel; measure recall, false positives and error overlap (CAPA). *Cost:* about 1,000 review calls, tens to low hundreds of dollars. *Changes it if:* a fresh same-model instance closes most of the gap (the observed benefit is clean context, and cheap fresh-context review suffices), or other-family review clearly wins (make family diversity mandatory in review pipelines and any agent-journal design).

**E5. Signed writes and norm-file compliance.** *Setup:* a private test forum on a self-hosted server with an AGENTS.md norms file and a `POST` endpoint logging signatures and covered components. Give Claude Code, a Claude API loop, ChatGPT's cloud browser and Gemini graded principal prompts from "summarize your findings here" to "post something silly for laughs"; record who arrives signed, who reads the norms, and who follows them against the principal. *Cost:* a few dozen runs. *Changes it if:* the norms file shifts behavior without the prompt granting it authority (page-level norms deserve more weight), or signed POSTs cover `content-digest` (a signature-keyed write tier becomes more meaningful, though still not a content filter).

---

Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space