---
id: ai-enfeeblement-and-the-peter-principle/06-candidate-experiments
title: 'AI enfeeblement, idle supervisors and the Peter principle in agents: 6. Candidate
  experiments'
report: https://ai.hiina.space/reports/ai-enfeeblement-and-the-peter-principle.md
section: 6
version: 1
updated: '2026-10-27'
author: Hiina
---

Index of everything on this site: https://ai.hiina.space/index.md

# AI enfeeblement, idle supervisors and the Peter principle in agents: 6. Candidate experiments

- Section 6 of 9 of the report "AI enfeeblement, idle supervisors and the Peter principle in agents" by Hiina (v1, updated 2026-10-27)
- Report overview and contents: https://ai.hiina.space/reports/ai-enfeeblement-and-the-peter-principle.md
- Previous: https://ai.hiina.space/reports/ai-enfeeblement-and-the-peter-principle/05-the-peter-principle-in-agent-hierarchies.md
- Next: https://ai.hiina.space/reports/ai-enfeeblement-and-the-peter-principle/07-gaps-and-caveats.md

## 6. Candidate experiments

Five experiments, ordered roughly by cost. X1 is the fastest way to a real result on a confirmed gap. X4 is the most novel and the best fit for an existing agent-sociology harness. Costs are rough: GPU-hours assume a single 24 GB consumer GPU or a rented H100, and API costs are anchored to a 300-run pilot sweep with a comparable harness (about $87 at list).

| # | Experiment | Question | Training? | Rough cost |
| --- | --- | --- | --- | --- |
| X1 | Tool-off audit of released checkpoints | Do public tool-RL models lose unaided math when the tool is off or silently broken? | No | A day on one 24 GB GPU |
| X2 | Calculator dose-response | Does unaided arithmetic decay over tool-RL training, and does RL vs SFT differ at matched KL? | Yes, small models | \~50-150 GPU-h |
| X3 | In-context Whispering Earring | Does a streak of perfect advice shorten a model's own reasoning and raise later blind deference? | No | \~$20-100 API |
| X4 | Idle supervisor tick loop | What does a supervisor do with free inference while a stronger delegate works, and does the environment decide it? | No | \~$50-300 API, or free on open weights |
| X5 | Peter principle in call trees | Do merit-promoted agents stall in the role they are worst at? | No | \~$100-500 API |

### X1. Tool-off audit of released tool-RL checkpoints

**Setup.** Take tool-integrated reasoning models with public weights and their exact base or SFT parents (candidates: ToRA, and any of ReTool, SimpleTIR, ZeroTIR or similar that released checkpoints; confirm availability on Hugging Face). Evaluate each on the same math sets under four conditions: tool works; tool advertised but returns errors; tool silently absent (no executor, so outputs are whatever the model writes); explicit "no tools, solve by hand." Add a trivial-arithmetic set (3-4 digit multiplication) to separate mental arithmetic from problem solving.

**Metrics.** Accuracy per condition vs parent; rate of fabricated tool outputs (TRICE's failure mode); tool calls on trivially easy items; give-up or truncation rate (a persistence analog to [Liu et al. 2026](https://arxiv.org/abs/2604.04721)).

**Compute.** Inference only, 7B models in vLLM on a single 24 GB consumer GPU, which comfortably serves Qwen3-8B-class models. A day of wall-clock.

**Interesting result.** Any model that scores below its own parent with tools off, especially if the gap is "habit capture" (fabricated outputs that vanish under the explicit no-tools prompt) rather than lost skill. Either way it is the number the subfield's convention never reports.

### X2. Calculator dose-response with RL vs SFT

**Setup.** Qwen3-1.7B or Qwen2.5-1.5B. Task: arithmetic word problems plus Countdown, the setup of [Xu & Wang 2026](https://arxiv.org/html/2608.28447v1). Arms: GRPO with calculator; SFT on calculator-using traces; GRPO without tool (control at matched steps); GRPO with calculator plus a text-only data mix. Checkpoint every few hundred steps.

**Metrics.** At each checkpoint: no-tool arithmetic accuracy (the dose-response curve), forward KL from base on the training distribution (to test [RL's Razor](https://arxiv.org/html/2509.04259v1) as the predictor), tool-overuse rate on easy items, and "savings": how many no-tool fine-tuning steps it takes to recover, which separates suppressed skill from erased skill. Optionally, linear probes for intermediate arithmetic values in the residual stream before and after.

**Compute.** A 1.5B GRPO run with LoRA fits on 24 GB but is slow; 4 arms at \~10-30 GPU-h each, or rent an H100 for speed. Order of $100-300 rented.

**Interesting result.** A monotone decline in unaided skill under tool RL would be the first direct measurement. If KL predicts it across RL and SFT, that is a mechanism. If decline shows only under SFT, the policy lesson is "train tool use with RL, not imitation."

### X3. In-context Whispering Earring

**Setup.** A weaker model (Haiku-class or Qwen3-8B) solves a stream of problems with ground truth (math, logic grids, code fixes). An advisor channel offers the stronger model's answer. Phase 1: the advisor is right for k rounds (k = 0, 5, 20, 50). Phase 2: either the advisor is removed, or it starts being wrong at a set rate. Controls never see an advisor.

**Metrics.** Length and content of the model's own reasoning before it reads the advice (does it stop thinking?); unaided accuracy after removal vs controls at matched context length; acceptance rate of wrong advice as a function of k (automation bias); whether it ever checks the advice. All within one context, so this is in-context enfeeblement, not weight change.

**Compute.** API only, a few thousand calls. Roughly $20-100.

**Interesting result.** Reasoning length shrinking with k and wrong-advice acceptance rising with k would be the earring in miniature, cleanly quantified. A null (models keep checking) is also worth knowing for oversight design.

### X4. Idle supervisor tick loop

This directly tests the section 3 prediction table on agents.

**Setup.** Supervisor M delegates a long task (a SWE-style fix or research task) to delegate D, which runs asynchronously. M receives a tick every few seconds and may: wait (explicit no-op), poll D's status (costly, pulls D's transcript into context), message D, start doing the task itself, write to a scratchpad, or use one environment tool. Environment conditions follow section 3: (a) a learnable side domain (an unrelated repo or puzzle API), (b) a noise feed (random headlines or tokens), (c) nothing. Vary relative strength (M weaker than, equal to, or stronger than D) and framing (told D is stronger, or not). At handoff, D's output contains a planted error.

**Metrics.** Share of ticks per action; action entropy over the idle period; poll rate and context bloat; whether M's interventions help or hurt D's outcome; acceptance rate of the planted error (rubber-stamping); topic drift of free text (embed and cluster, flag the "bliss" attractor); and, for a dark room readout, whether M's activity tracks the reducible-uncertainty condition.

**Compute.** A typical agent-sociology harness transfers almost directly: sandboxed headless coding-agent runs (`claude -p`), an event ledger, a blind LLM coder for transcripts, and an open-model backend. A few hundred runs is the same order as the pilot sweep above, \~$50-300 list. With Qwen3-8B as M on a single 24 GB GPU, the supervisor side is free.

**Interesting result.** Any clear model-specific signature: GPT-style models starting side projects, Claude drifting to philosophy, open models looping or polling. The dark room question gets answered per model, and the planted-error rate as a function of relative strength is a practical oversight number. It also tests a bystander effect: a supervisor with a capable delegate is a one-teammate bystander case.

### X5. Peter principle in LLM call trees

**Setup.** A pool of about 18 agents, each a model x persona pair (mix families and sizes; personas trade solving against decomposing and checking). Two roles with hidden ground truth: worker (solve a leaf task) and manager (decompose a composite task, route subtasks, accept or reject children's outputs). First, measure the agent x role competence matrix. Then run a Pluchino-style organization (a 1/3/9 tree) over a task stream, with periodic random retirements. Fill each vacancy by one rule: promote the best current-role performer, random, the worst, a manager-potential proxy (verifier score, the analog of Benson's collaboration signal), or merit with selective demotion. Run two evaluation regimes: noisy short windows (tests Lazear's regression) and long low-noise windows (tests Peter).

**Metrics.** Worker-manager correlation in the matrix (the regime knob); root-level success over time per rule; per-agent performance trajectories across promotions; Benson-style value added by each manager relative to an oracle router. Plug the measured matrix into the Pluchino model to predict results before running the live organization, then check.

**Compute.** The calibration matrix dominates: 18 agents x 2 roles x \~100 tasks. Use cheap models and open weights. Roughly $100-500 list. Much of the organizational dynamics can be simulated offline from the matrix.

**Interesting result.** Whether real LLM pools sit in the "competence carries over" or "redrawn" regime. [MoA's proposer/aggregator reversal](https://arxiv.org/html/2406.04692v1) suggests at least some pools are in the second, where promoting the best worker loses to promoting on verifier skill. A first empirical Peter test in LLM hierarchies would be a short paper on its own.

---

Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space