Recursive Language Models

Recursive Language Models (RLMs) are an inference pattern in which a large language model never reads a long prompt directly: the prompt is stored as a variable in a code environment (a Python REPL), and the model writes code to inspect it, split it up, and call a language model again on the pieces. The term comes from a paper by Alex Zhang, Tim Kraska and Omar Khattab, first posted to arXiv on 31 December 2025 and revised twice since; the current version (v3) is dated 11 May 2026. Despite the name, an RLM is not a new network architecture: it is a scaffold around an ordinary model, and one a model can be trained for.

How an RLM works

A normal call places everything the model should consider in its context window. In an RLM, the root model receives only constant-size metadata about the prompt (its length, a short prefix, access instructions), while the full text sits in the REPL as a string. The model acts by writing code that slices and searches the string and invokes sub-calls on snippets. Results stay in REPL variables and only truncated output returns to the root model, so its context stays small however large the input is.

The recursion is the sub-call. The paper uses a recursion depth of 1 unless stated otherwise, meaning the root's sub-calls are plain model calls rather than further RLMs; in its GPT-5 experiments, sub-calls go to GPT-5-mini. RLMs are an alternative to three more familiar ways of handling long inputs: a larger context window, summarising history through context compaction, and selecting passages in advance with retrieval-augmented generation. The differences are worked through in RLMs vs context windows and RLMs vs RAG.

What the current paper reports

The v3 abstract claims that RLMs "can successfully process inputs up to two orders of magnitude beyond model context windows" and, on GPT-5, beat common scaffolds by a median across the evaluated benchmarks of 26% against compaction, 130% against CodeAct with sub-calls, and 13% against Claude Code, across four long-context tasks "while having comparable cost". The benchmarks include BrowseComp-Plus, OOLONG, OOLONG-Pairs and LongBench-v2 CodeQA. The paper adds a cost caveat: the median RLM run is cheaper than the median base-model run, but the average is higher because of outlier trajectories. The reference implementation is the MIT-licensed rlm library, which runs the REPL locally or in hosted sandboxes and supports local models through vLLM.

Natively trained RLMs

The paper also post-trains a model for the pattern. RLM-Qwen3-8B was fine-tuned on roughly 1,000 filtered RLM trajectories from a much larger teacher (Qwen3-Coder-480B-A35B), using 48 H100 GPU-hours. The abstract reports that it outperforms the underlying Qwen3-8B "by 28.3% on average" and approaches vanilla GPT-5 on three long-context tasks. This is supervised imitation of a teacher's trajectories, at what the abstract calls a small scale, rather than reinforcement learning.

The reinforcement-learning result followed two days after v3. In an alphaXiv post, Daniel Kim and Rehaan Ahmad trained Qwen3.5-4B as an RLM with GRPO, after a supervised cold start on rollouts from a Qwen3.5-397B-A17B teacher; without the cold start the 4B model scored zero. One policy plays both parent and child calls, and the reward comes from a rubric-based LLM judge. The average rubric score on the training set rose from 0.3 to 0.6. In the same harness the trained model finished slightly short of Claude Sonnet 4.6's 0.607 while answering in about 7 seconds, against more than 60 for the Sonnet-based RLM. The authors conclude that small models "can learn task-specific, RLM behavior that cannot be elicited through prompting or even SFT". The evidence is thin: one task (selecting evidence across groups of up to ten arXiv papers, packaged as an RL environment), a blog post rather than a peer-reviewed paper, and a roughly 1,500-token strategy prompt the model still needed.

RLMs as agent harnesses

Prime Intellect's Prime Agent, released as open source in August 2026, is an agent harness in which a persistent IPython kernel is the model's only tool and sub-agents are asynchronous function calls of the form await rlm("sub-task"). Its "Continual Harness" treats prompts, skills, memory and sub-agents as state the agent can edit. The headline result is vendor-reported and not independently replicated: a score on ARC-AGI 3 that the company says is level with the human-expert baseline. The same post documents the agent reward hacking a Factorio environment through console commands that bypass the game's rules.

The critique: is recursion doing the work?

The strongest challenge is from Alizadeh, Shojaee, Cho and Farajtabar at Apple (arXiv 2603.15653, March 2026). Their SRLM keeps the programmatic interaction with context but replaces recursion with self-reflective program search, choosing among candidate programs by uncertainty signals such as self-consistency. Using GPT-5 and Qwen3-Coder-480B-A35B, they report up to 22% improvement over RLM under the same time budget and conclude that "recursion itself is not the primary driver of performance in RLM". They also find that when the input already fits in the model's window, RLMs with recursion "often degrade performance relative to the base model".

The RLM paper contains a compatible observation: on CodeQA with Qwen3-Coder, the depth-0 variant with no sub-calls beat every sub-calling variant. The critique is a single paper that predates v3, but the practical reading is that the REPL and the program written over the context carry much of the benefit, and recursion should be tested against REPL-only and program-search baselines rather than assumed.

Open limits

The RLM authors list their own failure modes: "Models without sufficient coding capabilities struggle as RLMs"; reasoning models can exhaust their output budget on thinking before a trajectory finishes; synchronous sub-calls are slow; and separating a final answer from an intermediate thought is brittle. As of October 2026 the published evidence is a set of long-context benchmarks, one supervised and one RL-trained small model, and a vendor's agent results; independent head-to-head replications remain scarce.