Synthetic Data

Synthetic data is artificially generated data that mimics the statistical properties and structure of real-world data—used for training AI models, testing software, and conducting analysis without exposing sensitive real data.

Synthetic data has become critical infrastructure for AI development. As models grow larger and more capable, the demand for training data has begun to exceed the supply of naturally occurring data. Estimates suggest that high-quality text data from the internet may be substantially exhausted by 2026-2028. Synthetic data—generated by AI models themselves—fills this gap: models generate training examples, which are filtered for quality and used to train the next generation of models.

The applications extend well beyond AI training. Healthcare organizations use synthetic patient records to develop algorithms without HIPAA concerns. Financial institutions generate synthetic transaction data for fraud detection testing. Autonomous vehicle companies create synthetic driving scenarios to test edge cases that rarely occur in real-world data but are critical for safety. NVIDIA's Omniverse generates photorealistic synthetic imagery for training computer vision models in manufacturing and robotics.

The quality of synthetic data has crossed an important threshold. Research demonstrates that models trained on carefully curated synthetic data can match or exceed the performance of those trained exclusively on real data for many tasks. Diffusion models generate photorealistic training images. Language models generate nuanced text datasets. The implication is a virtuous cycle: better models produce better synthetic data, which trains better models—a dynamic that may sustain AI capability growth even as organic data sources become constrained.

From synthetic examples to executable environments

For training AI agents, the unit of synthetic data has shifted from a static example to an executable RL environment: a generated task together with the files, tools or simulated services it runs against and a programmatic test that decides whether the agent succeeded. A model trained with reinforcement learning then produces its own trajectories inside the environment, and the verifier, rather than a human label, supplies the signal.

Two 2026 papers illustrate how little verified material this can take. EnvFactory (Xu et al., May 2026) generated 85 verified stateful tool environments across seven domains and reported gains for Qwen3 models of up to 15% on BFCLv3 and 8.6% on MCP-Atlas. Envs-FORGE (Wu et al., August 2026) rewrites each task's instruction, fixtures, reference solution, tests and Docker image together, steering difficulty toward what the current model can almost solve; only bundles that pass verification enter training. With 100 verified environments it improved Qwen 3.5 35B from 40.0% to 49.2% on the tb-core terminal benchmark and from 73.4% to 77.1% on SWE-bench Verified. Where writing executable environments is impractical, a learned simulator is an alternative: Alibaba's Qwen-AgentWorld (June 2026) trains language world models on more than 10 million real interaction trajectories to stand in for the environment. All three are author-reported results that have not yet been widely replicated.

Self-play: models that write their own tasks

In self-play, the model being trained also generates the curriculum. Absolute Zero (Zhao et al., May 2025) had one model propose code-reasoning tasks and solve them, using a code executor to validate both, with no external data. SPADE (Liu et al., August 2026) extends this to whole environments: a single model acts as an Environment Designer that writes long-horizon environments as code and as a Reasoning Agent that learns in them, and at 30B parameters the authors report an average gain of 5.3 points over the strongest fixed-environment baseline across eight held-out benchmarks. The paper is marked as work in progress.

Model collapse and data gating

The virtuous cycle has a documented failure mode. Shumailov et al. (2023) showed that training models on model-generated content "causes irreversible defects in the resulting models, where tails of the original content distribution disappear", and named the effect model collapse. The practical defence is the filtering step already implied above: deciding which generated items are admitted. In a controlled study of self-play RL, Pu et al. (May 2026) found a strict gate on generated tasks sufficient for stability under every reward variant tested, while no reward variant was sufficient once the gate was removed; they conclude that data-level gating, "not reward calibration, is the binding constraint on self-play stability". That finding comes from two controlled tasks and may not transfer unchanged to larger settings.

Some reported collapse is also an artefact of tooling. Liu and Han (September 2026) traced rapid loss of lexical diversity in recursive-training experiments to a fixed sampling seed shared across a batch in the vLLM inference server: with a seed per request, the unique-4-gram fraction of two test checkpoints stayed near 0.98 through three generations, while a replayed shared seed drove seven checkpoints to between 0.045 and 0.38. Verification, strict admission filters, fresh sampling and a continuing supply of real data are what keep synthetic pipelines productive.