In our batched-learning experiments, we achieved 100% across all 25 games in our ARC-AGI-3 agent-versus-human-baseline evaluation. The result came from evolving a shared skill across batches of games. It is a result on the games in that experiment, not a claim about unseen-game performance.
The more useful story is an architectural choice we made along the way. Should an agent learn across several games as it goes, or specialize on each game independently and combine the lessons afterward?
Both approaches can produce better skills. They put the burden of finding reusable knowledge in very different places.
What we are actually changing
Our research setup has three parts. A task agent plays using a domain skill and builds task memory. A learner reads evidence from those attempts and proposes edits to the domain skill and its own learning procedure. A fixed controller schedules sessions, records variants, and applies the evaluation rules.
The foundation model and evaluation rules stay fixed during the experiment. The editable material is external: instructions and supporting skill files. A candidate is a version of that material which we can evaluate and compare with earlier versions.
Fresh task attempts start without the previous player's task memory. That lets us ask whether the skill helps a new agent learn, rather than simply letting the old agent remember the solution. As discussed in the different levels of learning, we also need to keep solutions out of the shared skill itself.
Option 1: learn across batches
The approach we chose uses batches of three games to evolve one shared domain skill. Experience from several games informs the learner's revisions. Regression checks at batch boundaries test whether changes have damaged earlier capabilities.
The batch is a practical compromise. Replaying the accumulated games after every generation edit is expensive. Deferring those checks reduces evaluation work, at the cost of leaving some regressions undiscovered until the next checkpoint. The batch size controls how much change we allow between broader checks.
There is also a learning benefit. A procedure useful in game A has to coexist with what the learner encounters in games B and C. Conflicting evidence arrives while the shared skill is being formed. The learner must qualify, revise, or discard advice that is too specific.
Option 2: specialize first, consolidate later
The alternative ran independent learning loops on different games. Each branch had its own domain skill, learner procedure, memory, and archive of candidates. Local changes were evaluated only on that branch's game.
Once every branch reached its success threshold, a consolidator would read the successful local policies, reconcile them into one shared skill, and evaluate that skill alone on fresh attempts across the selected games. This resembles map-reduce: distribute the local work, then combine its outputs.
The appeal was elapsed time. Independent branches could learn concurrently. But local success did not guarantee that their procedures would form a useful shared skill.
Why we chose batches
Our main concern emerged from the experiments: a learner that sees only game A tends to write rules for game A. A local score rewards whatever helps on that game, including overly specific advice. The consolidation step then has to undo some of that specialization.
For example, “follow the red object” could be useful in one game and misleading in another. “Establish which object each control affects before planning a route” is a more general procedure. A learner exposed to multiple games has earlier opportunities to discover that distinction.
Our batch of three brings that filtering into each learning step. It encourages procedures that accommodate multiple games while they are being developed. It does not guarantee generalization, and we have not established that three is an optimal batch size. It was the practical choice for this experiment.
The independent design had two further problems. Its acceptance gate used the training games without the hidden validation set, so passing it would not establish transfer to unseen games. It also required 100% on every local game before consolidation could begin.
That second gate was especially brittle for human-relative action efficiency. Clearing a game and matching the human baseline are different requirements. A single stubborn game could prevent consolidation indefinitely, even when the other branches had useful procedures to contribute. Our experience with near-saturated scores and games that remained far below the target made that risk concrete.
We discarded independent local learning followed by consolidation as the main strategy. Parallel execution remains a scheduling tool we can consider where experiments are independent; it does not require separate per-game learning policies.
Research before productization
The research loop gives us controlled places to compare policies: fixed evaluation rules, recorded versions, explicit starting conditions, and regression results. Reproducibility also requires recording model and tool configurations, task selection, and resource use. A saved skill alone cannot explain a result.
Our product harness has a different operating goal: improve while doing useful work. It preserves task progress, introduces local and shared reviewers when useful, and can request research when experience leaves a question unresolved. Its recursive runtime supports that work. The experiments help us decide which learning behavior deserves to become part of the product.
One idea survives from consolidation: reconciling conflicting procedures is part of the learner's job throughout a run. Another remains a research proposal: periodically distill a skill and begin a fresh run with empty learner memory and archive, preserving useful guidance while testing how much accumulated history it still needs.
We are now experimenting with simulated Luma acceptance: yes/no predictions with explicit confidence, evaluated against labeled examples. It changes the feedback problem from interactive game play to classification. That gives us another setting in which to test the question that started this work: does experience produce a better way to approach the next task?