Model Collapse

Model collapse is the degenerative process in which generative models trained on data produced by earlier generations of models progressively lose information about the true data distribution — first the rare cases in its tails, eventually most of its variety. The term was introduced by Shumailov et al. (preprint May 2023; Nature, July 2024). The effect is real and reproducible under the conditions those authors studied. Subsequent work has shown that those conditions are narrower than the popular reading suggests: whether collapse happens depends mainly on whether real data is kept and on what is allowed back into the training set.

The Original Result

Shumailov and colleagues describe two stages. In early collapse, low-probability events disappear from what the model generates. In late collapse, the model converges on a distribution with sharply reduced variance that bears little resemblance to the original. They attribute this to three compounding error sources: statistical approximation error from finite sampling, functional expressivity error from the limits of the model class, and functional approximation error from the learning procedure. The effect was shown for Gaussian mixture models, variational autoencoders and language models.

The language experiment fine-tuned a small model (OPT-125m) on Wikitext2, then trained each generation on text sampled from the previous one. Perplexity worsened over generations, and later generations emitted text the original model would never have produced — the paper's example output degenerates into a repetitive list of jackrabbit varieties. A second setting preserved 10% of the original data in each generation, a design choice that anticipates the later debate.

When It Does and Does Not Happen

Study	Setting	Finding
Alemohammad et al., July 2023	Image generators in three kinds of self-consuming loop	Without enough fresh real data in each generation, quality or diversity progressively declines
Shumailov et al., July 2024	Each generation trained on the previous generation's output	Tails vanish, then variance collapses
Gerstgrasser et al., April 2024	Synthetic data accumulates alongside the original real data	Collapse is avoided; in a linear-model analysis, test error has a finite bound independent of the number of iterations
Dohmatob et al., October 2024	A fixed fraction of synthetic data in the training set; theory plus language-model and image experiments	As little as 1% synthetic data can stop larger training sets from helping ("strong" collapse)
Schaeffer et al., March 2025	Position paper surveying the literature	Eight distinct, sometimes conflicting definitions in use; many predicted collapse scenarios are "readily avoidable" under realistic conditions

These results conflict less than they appear to. The replace-everything loop that produces dramatic collapse is not how training corpora are actually assembled, and keeping the real data largely defuses it. But "largely" is not "entirely": Dohmatob et al. show a weaker form in which synthetic contamination caps the benefit of more data even when real data is retained. The honest summary as of October 2026 is that total collapse is avoidable with ordinary data hygiene, while a quieter loss of tail coverage from unfiltered synthetic data remains a live concern and is harder to measure.

One 2026 finding is a caution about the experimental literature itself. Liu and Han (September 2026) report that when an inference server reuses one sampling seed across all requests in a batch and across generations, recursive training amplifies the replayed randomness: the fraction of unique 4-grams fell to between 0.045 and 0.38 by the third generation on the StableLM checkpoints they tested, but stayed near 0.98 when each request got its own seed. Some reported collapse, on this account, is an artefact of the sampling setup rather than of recursion as such. This is a single recent preprint.

Why It Matters for Agent Training

Model collapse stopped being only a question about the future web when training on model-generated data became deliberate practice. Distillation, synthetic task generation and self-play all feed a model's outputs, or a sibling model's outputs, back into training. In reinforcement fine-tuning the feedback loop is tighter still: the model generates the trajectories, and in self-play it also generates the tasks.

Data Gating as Mitigation

The mitigation with the most direct support is controlling what enters the training set, rather than adjusting the objective. "Survive or Collapse" (Pu et al., May 2026) varied both the reward design and the strictness of a data-level filter in self-play RL on code-prediction and domain-specific-language tasks. A strict gate was sufficient for stability under every reward variant tested; with the gate removed, no reward formulation prevented collapse. The authors call data filtering "the binding constraint on self-play stability". Their setting is reinforcement self-play on narrow tasks, not web-scale pre-training, so it supports gating as a mitigation for self-generated training loops specifically.

Other measures follow from the earlier studies:

• Keep the real data. Accumulate synthetic data alongside the original corpus rather than replacing it (Gerstgrasser et al.).
• Filter with something external. A verifier, test suite or held-out check is a gate the generating model cannot quietly satisfy; a filter that is the same model's own judgement is weaker.
• Track provenance. Knowing which training data is model-generated is a precondition for limiting its share.
• Measure diversity, not just accuracy. Pu et al. observe training metrics diverging before validation accuracy fell; tail loss shows up in distributional measures first.