An agent opens an unfamiliar puzzle. It presses a button, watches a red object move, and writes down what happened. A few attempts later, it can finish the level.
What did it learn?
It might have learned a fact about this puzzle. It might have discovered a better way to investigate unfamiliar controls. Or it might have improved the procedure it uses to judge its own investigations. Those changes have different lifetimes, different evidence requirements, and different value on the next task.
Our approach separates them. The foundation model stays fixed; experience changes external memory and written procedures that future agents can use. As we argued in Agents Shouldn't Start Over, useful experience should survive the session that produced it. The harder question is what deserves to survive.
Learning what happens
Start with the puzzle. The agent believes a button moves the red object one square to the right. It predicts a movement, presses the button, and compares the next observation with its prediction.
If the object does not move, the belief needs work. Perhaps a wall blocked it. Perhaps the control affects a different object. Perhaps movement depends on a hidden mode.
This is learning a world model: an account of how the environment behaves. Its contents belong with the task. “The red object moves right when this button is pressed” may be valuable throughout this game and useless in the next one.
A familiar human parallel is learning where the light switch is in a new room. Remembering it saves effort next time. It does not establish where every room's switch must be.
Learning how to investigate
Now suppose the agent keeps pressing several controls together. It sees changes, but cannot tell which action caused which effect. More observations accumulate without resolving the uncertainty.
The problem has moved beyond a mistaken belief. The investigation procedure is producing ambiguous evidence.
A useful revision might be: “When actions are cheap and reversible, change one control at a time and compare the before-and-after state.” The next puzzle can benefit from that instruction without inheriting the first puzzle's mechanics or solution.
This resembles learning a study or debugging habit. Someone can memorize the answer to one exercise, or discover a way to work through unfamiliar exercises. Both are learning, but only the latter is a plausible method to carry across different problems.
The conditions matter. Isolating variables may be practical in a resettable puzzle and impossible in a changing social environment. A transferable procedure should say when it applies; “always do this” is often an overgeneralization.
Learning how to improve the method
Suppose a reviewer keeps recommending new procedures after a single failed action. The player becomes inconsistent, abandoning approaches before it has collected enough evidence to judge them.
Here, the review method itself needs attention. A possible revision is to distinguish an unsupported belief from a recurring defect in the way evidence is gathered. Another is to ask what observation would show that a proposed change helped.
The human analogy is reflection on one's learning habits: noticing that constantly switching study techniques prevents a fair assessment of any of them. This is an intuitive comparison, not a claim that our architecture reproduces the brain.
Nor does allowing a reviewer to edit its instructions prove that it becomes better at reviewing. A convincing explanation is a candidate improvement. Its consequences still need to be observed and measured.
Where these ideas live
In our product harness, the task agent maintains its world model. Local reviewers examine learning within a task; a shared reviewer considers procedures that could help across tasks. Their review methods are separate from the guidance given to the player. An on-demand researcher investigates unresolved questions when ordinary experience is insufficient.
Our research setup groups these responsibilities differently: a learner edits the domain skill and its own learning procedure within a controlled evaluation loop. Consequently, labels such as L1 and L2 do not refer to identical components in both systems. The useful distinction is what changes: task knowledge, player guidance, or the method for improving that guidance.
These responsibilities do not require an endless tower of agents. Recursive execution helps organize work; persistent learning requires deciding which changes to retain and why.
Carry the method, test the transfer
A fresh agent with empty task memory is useful for evaluation, but it is not enough. If its shared skill contains the route through a training puzzle, that agent has inherited the answer through another channel.
We therefore want shared guidance to carry procedures rather than game solutions. Task facts remain local. Methods retain their conditions and supporting evidence. Claims about transfer require evaluation beyond the situations that produced the change.
This separation also suggests a practical rule: revise a belief when a prediction fails; reconsider the learning method when the way of learning repeatedly fails to resolve the problem.
Our batched-learning experiments put that distinction under pressure. They ask whether a shared skill can help agents learn across games, instead of becoming a collection of answers to individual games. That is the difference we want our experiments to measure.