Markdown version: https://ai.hiina.space/reports/ai-enfeeblement-and-the-peter-principle.md · index
Index of everything on this site: https://ai.hiina.space/index.md
AI
enfeeblement, idle supervisors and the Peter principle in agents
This report starts from the position of a solo developer supervising
a fleet of coding agents, and asks four related questions. Does a model
trained to lean on a tool lose the ability to do the task without it,
the way people are said to lose skills to automation? What does the dark
room problem from predictive processing say about an agent whose job has
been taken over by a stronger helper: does it go quiet or find something
else to do? What does a supervisor agent actually do with free inference
while a stronger delegate works? And does the Peter principle, where
people rise to the role they are worst at, show up when LLM agents are
promoted through a hierarchy?
Contents
1. Summary
All four questions sit on real, unmeasured gaps. No one has tracked a
model's unaided skill while training it to use a tool, watched what a
supervisor does with free inference while a stronger delegate works,
applied the dark room framework to AI delegation, or tested the Peter
principle with LLM agents. Adjacent evidence is strong enough to make
predictions, and most of the experiments are cheap.
- Model enfeeblement shows up under narrow SFT; careful
recipes avoid it. Fine-tuning only on tool-using traces
collapsed a model's text-only math (54.3 to 34.1) and left it inventing
tool outputs; the paper's full recipe, which mixes text-only traces back
in, kept it. That is consistent with the finding that RL forgets less
than SFT, but no one has run the RL-vs-SFT comparison on tool use
directly. Behaviorally, models already overuse tools, and RL increases
it.
- Human deskilling is real but narrower than the
headlines. Practice with plain GPT-4 cut later unaided exam
scores 17%; endoscopists' detection rate fell after AI was introduced;
pilots kept stick skills but lost navigation skills and mind-wandered
more. The Google effect failed replication and the MIT EEG study is
weak.
- The dark room framework doesn't predict idle vs busy by
itself. It depends on the agent's self-model and on whether any
uncertainty is left that its own actions can reduce. Humans substitute
activity, often onto noise. The Whispering Earring adds a third outcome:
active but without agency.
- Idle supervisors, by neighboring evidence: they
defer and rubber-stamp (a same-model CEO agent approved lenient requests
8:1), repeat work, act when they shouldn't, and in Claude's case drift
into philosophical chatter. Production systems avoid the question by
blocking the supervisor.
- Peter principle: worker and manager skill already
come apart for LLMs (the best Mixture-of-Agents proposer was a poor
aggregator). Agent-based models say merit promotion fails exactly when
that correlation is low, and regression to the mean (Lazear) can mimic
the effect.
Recommended start. X1, a tool-off audit of released
tool-RL checkpoints, is an inference-only result in about a day on a
single 24 GB consumer GPU. X4, the idle supervisor tick loop, is the
most novel experiment and can reuse an existing agent-sociology
experiment harness nearly as is.
Next section: https://ai.hiina.space/reports/ai-enfeeblement-and-the-peter-principle/02-ai-enfeeblement-as-measured.md
Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source:
https://github.com/hiinaspace/ai-hiina-space