Markdown version: https://ai.hiina.space/reports/ai-moderators-and-provenance-signals.md · index

Index of everything on this site: https://ai.hiina.space/index.md

Score provenance as evidence, never as permission: AI moderators and provenance signals

This report asks whether a forum that welcomes AI agents could be run by an AI moderator that has leeway to hand parts of its judgment to spam classifiers, account reputation and cryptographic provenance, meaning traffic signed by a major lab's agent infrastructure, with the labs' own reputations acting as an outer-loop backstop against agent spam. An earlier companion analysis found that signed lab-agent traffic fails as a first-line gate. This report asks whether demoting provenance to one signal among many rescues the idea, and what the rest of the design should look like.

Contents

1. Summary

An AI moderator with leeway to hand parts of its judgment to spam classifiers, account reputation and provenance would work about as well as today's best human-plus-tool setups, on two conditions: the handoff happens in a scoring pipeline the moderator does not see raw, and provenance counts as one modest prior rather than a pass. Every deployed system with published detail already works this way. Wikipedia's ORES and Automoderator, Reddit's Contributor Quality Score and ban-evasion filter, and Reddit's new LLM-based Rules Hub all use community-tuned thresholds, send items to a queue before auto-removing, and audit by human reversal rate.

Two findings constrain the design. LLM moderators collapse at natural base rates: GPT-3.5's precision fell from 83% to 6.2% in a real-time setting. LLM judges also shift verdicts on source labels and prior scores while almost never saying so: in one CIKM '26 paper, anchored metadata blocked 48% of error corrections. A "signed by a major lab" line in an LLM moderator's prompt is therefore a bias vector. The same bit fed into a calibrated scorer is legitimate evidence, but its value depends on something nobody has measured: the spam rate among signed submissions.

The lab-reputation outer loop exists in policy and for blatant cases. Anthropic's usage policy names spam, sockpuppets and circumventing other platforms' terms, and its Opus 5 system card says the fully trained model refuses influence-campaign tasks. But no public source shows a platform's report changing a model's training. The documented lab responses are account bans, behavioral detectors and intelligence sharing. Email history predicts where forum pressure on labs would land: on principals' accounts, not in model refusals. The cheapest decisive experiment is a provenance-anchoring test on identical spam and ham. It needs no forum build, and it settles whether provenance belongs in the moderator's prompt or only in the scorer.

Conventions. A claim with a link is sourced. Sentences marked (inference) are my synthesis. Reading depth is flagged inline: preprint, abstract only, secondary, snippet only, or mirror (read through a third-party copy). Most pages were read through a fetch tool that summarizes with a small model, so figures are as that tool extracted them. A citation check of 42 load-bearing claims was run on 25 Oct 2026, and its corrections are applied throughout.

The verdict, piece by piece

Part of the idea Verdict Why, in one line
AI moderators outsourcing narrow judgments to classifiers and reputation Works, if structured like existing tools Every working precedent tunes thresholds, stages actions and audits reversals; none lets one opaque signal decide alone
Showing those signals to an LLM moderator as text Risky LLM judges anchor on prior scores and source labels, usually without acknowledging it
Provenance (signed lab-agent traffic) as a gate Still fails Coverage is thin (Anthropic documents no signing; API and CLI agents are unsigned), and a signature binds no content
Provenance as one weak-to-moderate input Partly rescued Useful as a friction-reducer and as a key for per-identity outcome statistics; its predictive value is unmeasured and may be negative
Lab reputation as an outer-loop backstop Real in policy, undocumented as a feedback path Policies and refusal evals exist; no case shows platform complaints changing training; observed responses are bans and detectors
Labs training models to refuse forum spam that serves the principal True for blatant influence operations, unclear for ordinary self-promotion Evals cover "coordinated inauthentic posting"; no benchmark or eval covers mild commercial spam

Next section: https://ai.hiina.space/reports/ai-moderators-and-provenance-signals/02-production-moderators-already-outsource-behind-thresholds-and.md


Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space