An agent does not need to hold everything in its context window to work with it. It needs a way to inspect information, ask focused questions, and keep track of the answers.
That is the useful idea behind Recursive Language Models, a paper by Alex L. Zhang, Tim Kraska, and Omar Khattab. It changes how a model accesses a large input. The input becomes something the model can operate on through code.
For our work at Recursa, this is an execution foundation. Our larger goal is to make agents improve through experience. Before an agent can learn from a long history of work, it needs a practical way to examine that history.
A context window is a workspace
Imagine asking an agent to analyze several years of customer-support conversations: which problems increased after a product release, and what evidence connects them to the change?
Pasting the entire archive into a prompt creates two problems. It may exceed the model's context window. Even if it fits, finding and relating the right details can become harder as the input grows. Summarizing everything first also makes a decision too early: which details can safely be discarded before we know what matters?
The RLM approach keeps the original archive available outside the model's immediate context. In the paper's implementation, it lives in a variable inside a Python REPL: a programming environment that retains variables between code executions. The model receives information about that environment and writes code to explore it.
Inspect, interpret, combine
Our support example might unfold in four steps.
- Inspect the archive. The model writes Python to examine a few records and identify their structure: dates, product versions, conversation IDs, and message text. It can count records without reading every conversation into its own context.
- Select useful slices. Code separates conversations before and after the release. The model can inspect samples or search for candidate issues, then refine its selection as it learns more.
- Ask focused semantic questions. A smaller model call examines a manageable batch: “Which conversations describe a login failure? Return the conversation IDs and supporting excerpts.” This handles meaning that a keyword search might miss.
- Combine and check. Python aggregates the returned labels. The root model examines the findings, requests additional evidence where needed, and writes an answer linked to the original records.
The division of labor matters. Code is useful for counting, filtering, joining, and preserving exact identifiers. Model calls are useful for interpreting ambiguous language. Neither has to do the other's job.
The workflow also depends on the question. Finding one incident may require only a few excerpts. Estimating how common a problem is requires systematic coverage; searching for a convenient keyword would bias the result. An RLM provides a way to organize that work, but the model still has to choose a sound method.
What makes it recursive?
The model can construct a smaller problem from the input and invoke a model to solve it. In the general formulation, that smaller problem can use the same mechanism again. The root then uses the returned result to continue its own work.
The distinctive part is the combination of subcalls with external context. Delegating tasks alone does not remove the root model's input limit. An RLM can manipulate a large input before deciding which parts any model call should see.
There is an important boundary between the idea and the experiment: the paper evaluated a maximum recursion depth of one. Its root model could make subcalls, but those calls were ordinary language-model calls. The results do not require an arbitrarily deep tree of agents. See the paper's limitations section.
Useful, with tradeoffs
The paper reports handling inputs above ten million tokens on its long-context evaluations. That demonstrates a useful change in what the system can access, without enlarging the underlying model's context window. It does not mean every task becomes easier or cheaper. See the results and discussion.
Extra calls introduce overhead, and model decisions can be wasteful. The authors observed repeated checking and unnecessary subcalls. Costs varied substantially across runs, with expensive outliers. On shorter inputs, a direct model call could perform better. Even the version with external context but no subcalls sometimes beat the full RLM.
Where learning enters
An RLM changes how a task is executed. It does not, by itself, preserve better procedures for the next task or improve the model's weights.
Persistent learning adds another loop: examine what happened, propose a better procedure, evaluate it, and retain changes supported by evidence. Recursive execution can help inspect traces and delegate focused reviews. The rules governing which changes survive must come from the learning system.
That distinction guides our architecture. In the next post, we separate facts, methods, and improvements to the learning process itself. In our game experiments, we examine how shared procedures evolve when they must work across several tasks.