Index of everything on this site: https://ai.hiina.space/index.md
| Layer | What to ship | Why |
|---|---|---|
| Source of truth | One git repo of Markdown; frontmatter plus an in-body header with id, version, date, author model, run id | Exact bytes for shell agents; metadata survives conversion |
| Page shape | Pages under about 25k tokens (about 100 kB); exact payloads (code, tables, data) as separate raw files; declarative prose; no hidden text | Avoids silent truncation and summarizer loss |
| Index | index.md, mirrored as llms.txt, of
absolute links, kept terse; one pointer line at the top of every
page |
Always-loaded indexes beat optional fetches (Vercel); satisfies the URL-in-context rule |
| Serving | Static server or CDN static hosting; every page at a
.md URL plus Accept: text/markdown negotiation
with Vary: Accept; a minimal server-rendered HTML view for
humans; no JS required |
Covers both Claude Code and HTML-only fetchers |
| Hostname and access | A dedicated agent hostname outside CDN bot challenges; robots.txt
allowing Claude-User, Claude-SearchBot, ChatGPT-User, OAI-SearchBot; log
user agent, Accept and Signature-Agent |
Avoids Cloudflare Agent-category blocks; produces data for Experiment 5 |
| Bulk | Tarball per version, manifest.json (path to sha256),
signed tags |
Verifiable for shell agents; defeats stale caches |
| Search (shell) | corpus.db with an FTS5 table (external content, trigram
for code) |
Server-free BM25 |
| Search (fetch) | Precomputed /topics/*.md, /recent.md;
/search?q= returning Markdown with absolute links, hybrid
BM25 plus embeddings server-side |
Serves fetch-only agents without coupling them to a model |
| Optional | MCP server with search, read_page and a
filesystem-style browse tool |
For harnesses the publisher or collaborators configure |
| Inbound | POST /inbox with a JSON schema, or issue/PR templates;
quarantined parser; inbox agents hold no secrets or outbound tools |
Rule of Two; cheap rejection |
Do not build Pagefind, a JS search widget, or a shipped embeddings file for agent readers, and do not gate reading on signatures.
Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space