---
id: llm-evolution-and-meta-optimization/01-summary
title: 'LLM evolution and meta-optimization: 1. Summary'
report: https://ai.hiina.space/reports/llm-evolution-and-meta-optimization.md
section: 1
version: 1
updated: '2026-10-27'
author: Hiina
---

Index of everything on this site: https://ai.hiina.space/index.md

# LLM evolution and meta-optimization: 1. Summary

- Section 1 of 10 of the report "LLM evolution and meta-optimization" by Hiina (v1, updated 2026-10-27)
- Report overview and contents: https://ai.hiina.space/reports/llm-evolution-and-meta-optimization.md
- Next: https://ai.hiina.space/reports/llm-evolution-and-meta-optimization/02-a-model-genome-exists-only-as-separate.md

## 1. Summary

One part of the biology maps more sharply than expected: lineage is a hard constraint. Weight merging, task arithmetic and subliminal trait transfer all need a shared pretrained ancestor ([Cloud et al., subliminal learning](https://alignment.anthropic.com/2025/subliminal-learning/)). In practice, "same base model" is the operational definition of a species. Text-level distillation crosses any lineage boundary, and that is what the 2025 to 2026 lab disputes are about. Anthropic's February 2026 post alleges more than 16 million exchanges from three labs, about 13 million of them from MiniMax alone ([Anthropic](https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks)).

On meta-optimization, can hyperparameters be tuned by gradient descent? Yes, millions of them ([Lorraine et al. 2019](https://arxiv.org/abs/1911.02590)). The outer gradient degrades as the training horizon grows, though ([Wu et al. 2018](https://arxiv.org/abs/1803.02021)). At LLM scale, practice has moved to tuning small proxies and transferring the settings through parameterization theory (muP). Higher up the stack, the methods that work are evolution strategies, populations and archives, and LLM-guided code mutation. "Evolving evolution" also works: learned ES and GA operators ([LES](https://arxiv.org/abs/2211.11260), [LGA](https://arxiv.org/pdf/2304.03995)) and LLM-written training objectives ([DiscoPOP](https://arxiv.org/abs/2406.08414)). Learned optimizers have become cheap to meta-train, but no one outside the authors has shown them winning at scale. The modded-nanogpt speedrun went from 45 minutes to 0.665 minutes across 92 records, five of them co-credited to self-labelled "AI System" entrants ([README](https://github.com/KellerJordan/modded-nanogpt)). It is a public evolutionary record that nobody has analysed as one.

Rigorous work on levels of selection is thin. There is a philosophical exchange (Hendrycks versus Boudry and Friederich), a Price-equation formalism its own author calls notational ([Frank](https://arxiv.org/pdf/2507.18549)), and one real empirical study of inheritance on Hugging Face ([Laufer, Oderinwale, Kleinberg](https://arxiv.org/pdf/2508.06811)). No one has built a data-driven ecology of labs, funding and talent. For family trees, good instruments exist: output phylogenies, provenance tests, weight fingerprints, and a $34 one-token census of 165 models. Nobody has combined them into a typed lineage graph, and harness genealogy is untouched. For a single 24 GB consumer GPU and a small API budget, the best returns are in analysing public evolutionary records (the speedrun log, the Hugging Face model graph) and in small-model lineage experiments.

How to read this report: sourced claims carry inline links. My own reasoning is marked "Inference:" or sits under headings that say so. "(abstract-only)" means this review read only an abstract, listing or blog summary, not the paper body. All pages were read through a summarizing fetch tool, and two verification passes re-checked the most load-bearing numbers against primary pages; their corrections are applied here. Most quantitative results below are self-reported by the proposing authors.

---

Text licensed CC BY 4.0 (attribution: Hiina, ai.hiina.space). Source: https://github.com/hiinaspace/ai-hiina-space