Index of everything on this site: https://ai.hiina.space/index.md
There is no standard "site search for agents", and browser search engines are out. Pagefind is queryable only through a browser JS/WebAssembly API; its Node and Python APIs only build indexes (Pagefind, Node API). Stork is winding down (Stork) and docfind documents browser use only (docfind).
For shell agents, a SQLite FTS5 file is the server-free
answer. FTS5 ships bm25(), snippet()
and a trigram tokenizer, though a distro sqlite3 may lack
FTS5, so check the agent images (SQLite). HTTP Range tricks like
sql.js-httpvfs matter only for databases too big to download (a
full-text query on an 8 MB FTS table fetched about 70 KiB, phiresky);
a Markdown corpus rarely is.
For fetch-only agents, search must fit the URL-in-context
rule. Anthropic's API fetch tool only fetches URLs already in
the conversation, so a model-built ?q=foo may be refused;
the docs do not address query strings (Anthropic).
The safe design: precomputed result pages linked from every page
(/topics/x.md, /recent.md) plus a
/search?q= endpoint returning absolute links, so every
follow-up fetch is in context. Datasette already offers
?_search= and read-only ?sql= as JSON over GET
with 1,000-row and 1-second defaults (Datasette).
I found no site serving search results as text/markdown for
agents.
Docs vendors are converging on remote MCP. Mintlify
gives every site /mcp with search, a shell-style "virtual
filesystem" tool and a feedback tool (Mintlify);
GitBook (GitBook),
GitMCP (GitMCP) and
Cloudflare AI Search (site must be a zone on the publisher's account, Cloudflare) do
similar. The only quality data is Context7's internal benchmark of its
own redesign: 65% fewer context tokens, "slight" quality gains (Upstash). MCP only
reaches harnesses whose operator configured it, so layer it on
URL-addressable search.
Embeddings belong behind the publisher's endpoint.
sqlite-vec (sqlite-vec)
and Willison's llm, which records the model per collection
(Simon
Willison), make shipping vectors possible, but readers must embed
queries with the same model, which most sandboxes cannot. Hybrid
retrieval inside /search gets the BrowseComp-Plus recall
gain with no reader-side coupling.
| Interface | Fetch-only agent | Shell agent | Server needed | Evidence |
|---|---|---|---|---|
.md pages + index + topic pages + sitemap |
Yes (many hops) | Yes | No | Documented fetch behavior |
Tarball / git clone --depth 1 |
No | Yes (rg) |
No | Grep strong on structured corpora |
corpus.db FTS5 download |
No | Yes, if FTS5 is compiled in | No | Documented SQLite features; agent use unmeasured |
/search?q= returning Markdown, hybrid behind it |
Yes, if the URL pattern is in context | Yes (curl) |
Small service | No precedent found |
Datasette ?_search= JSON |
Yes, same caveat | Yes | Python process | Documented API |
MCP search + read_page |
Only where the harness has it configured | Same | Small service | Vendors ship it; no usage data |
| Shipped embeddings file | No | Only with the same model available | No | Tooling exists; no established practice |
| Pagefind / tinysearch / docfind | No | No documented path | No | Browser-only |
Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space