When We Talk About Exploration, What Are We Really Evaluating?
From “Capability Stock” to “Capability Growth”
By Ming Zhang and Maxm Pan· Fudan University & Hunyuan Team, Tencent
TL;DR
Most benchmarks so far measure a model's capability stock. A more important question is whether a model placed among completely unfamiliar rules can gain new capabilities by interacting with and exploring its environment.
Learning by doing is one of the most effective ways people take on complex real-world tasks. An engineer chasing down a tenth bug understands the system architecture far better than when fixing the first. A scientist, too, has to search the literature, propose hypotheses, run experiments, and revise those hypotheses before there is any chance of cracking a hard scientific problem. The engineer relies on accumulated experience, and the scientist on exploring the unknown. A reliable long-running agent needs the same ability: it must turn what it has already done, and the feedback it got, into capability for next time.
So beyond what a model knows right now, there is a more fundamental question: as it keeps interacting with its environment, can a model with frozen parameters become smarter on its own?
We did a lot of research on static evaluation early on, and it became increasingly clear to us that static evaluation, at its core, measures the capability a model already has, its capability stock, which is largely a readout of what is already in its weights. Whether a model can work reliably in the real world over the long run depends on whether it can grow stronger capabilities at test time, which we call capability growth.
Over the past two years we turned our attention to capability growth and built EvaLearn and CL-bench, two benchmarks that were quite forward-looking when they came out. Now we are building on them with ExplorationBench.
From Experience to Context
A quick look back first.
EvaLearn, our NeurIPS 2025 paper, set out to break the independent-and-identically-distributed (i.i.d.) assumption in evaluation. We organized 648 problems into 182 sequential task chains and let models consult their earlier attempts and the environment's feedback. The results ran against intuition. Models with top static scores were not necessarily the best online learners. Some improved steadily along a sequence, while others got worse as noise and mistaken experience piled up in their history, a clear case of negative transfer.
So this kind of sequential learning ability deserves to be evaluated on its own. The work still left a puzzle, though: did a model actually understand the underlying rules, or had it only memorized the shape of the problems?
Answering that meant laying out explicitly what the model was supposed to learn. In CL-bench, released early this year, we put the material to be learned directly into the model's context, including fictional legal systems, complex professional procedures, and empirical laws hidden in experimental observations. With the context stripped away, top models solved under 1% of the tasks. Even with detailed, self-contained context, ten frontier models averaged only 17.2%, and the strongest reached just 23.7%.
These results cut against the optimism around context engineering. Many people put enormous effort into retrieval, reranking, and text compression, assuming without much thought that once information is in the window, the model will "learn" it. What we found is that getting information into the context window and having the model learn and use that knowledge are two very different things.
Every rule and fact in CL-bench's contexts was prepared in advance by human experts as correct, complete, ready-made material. But is the real world like that? Consider scientific research. Facing an open problem, researchers usually have no manual full of standard answers, and no way to know in advance which data to collect or what to measure. They can only keep proposing hypotheses, testing them in experiments, and distilling regularities from what they observe. What if a model faced the same situation?
From Context Learner to Explorer
ExplorationBench is designed for exactly this situation. The model gets only a sandbox it can interact with, built around an unfamiliar world, and no ready-made knowledge it can use directly. It has to propose hypotheses, design experiments, read the execution feedback, and then work out the rules of that world and check them.
Evaluating exploration ability, though, runs into two tricky problems. First, if we pick an established field as the target, then however hard the problems are, a model can recite answers from its pre-training data, and contamination is almost impossible to rule out. Second, if we pick open questions from the scientific frontier instead, there is no timely way to tell right from wrong, because testing a genuinely new hypothesis in physics or biology can take costly experiments and years of peer review.
So we built two "Alien Worlds", the unfamiliar worlds mentioned above, whose rules are counterintuitive yet fully checkable by formal means. The models have never seen these rules, yet the evaluator can execute them exactly and judge every answer precisely.
In AlienCode, the syntax looks ordinary, but 31 of its rules have been quietly changed (we call them discovery targets). SHATTER sounds destructive but actually multiplies. EMIT(100) prints 127, because integer literals are silently XOR-ed with 27. The manual says PLUCK counts positions from zero, yet it actually counts from one. The language also leaves out any addition primitive on purpose, so a model that wants to add has to assemble addition from other counterintuitive operations.
In AlienLogic, we modified 24 inference rules of natural deduction. Some steps that traditional logic takes for granted fail here, and some moves that used to be invalid now go through. Among its 70 held-out tasks we also planted theorems designated as unprovable, where a model scores only by recognizing the dead end and declining to prove them.
Across the two worlds there are 55 discovery targets and 140 held-out tasks, and every verdict comes from a program. Code is run by an interpreter, proofs are checked by a proof checker and a bounded certifier, and no LLM acts as judge.
The ExplorationBench evaluation protocol. Every system starts from the same flawed manual and the same fixed worked examples and chooses its own probes over four rounds. At every milestone, a copy of the conversation with tools disabled reports the rules the system believes hold and answers the held-out tasks.
The protocol runs four tightly controlled rounds of exploration. Inside the sandbox, the model submits probe code or proofs through a dedicated tool, and the environment immediately returns deterministic feedback with no explanation. After each round, the harness copies the conversation and cuts off tool access. Working fully closed-book, the copy states the rules it believes hold and answers every held-out task three times. The copy is then discarded, so test items never enter later exploration.
To account for chance and for how general the results are, every system runs three independent trajectories in each sandbox, and we follow how its competence changes from the worked examples (M0) to the end of four rounds (M4). Because exploration depends heavily on chance, we take Best@3, the best endpoint of the three trajectories, as the main result. We also report their mean and every individual trajectory.
Feedback Is More Effective Than Thinking Alone
Across ten frontier systems, exploration produced knowledge that is absent from their training data. After the worked examples, no AlienCode trajectory exceeded 15.7%; after four rounds, the best trajectory reached 89.0%, and seven of the ten systems passed 60%. AlienLogic's patches keep part of standard natural deduction intact, so it starts higher, and Best@3 rose from 33.8–51.9% to 59.5–81.0%.
Still, a higher score cannot simply be credited to exploration. A fair question is whether the gain came from the evidence exploration gathered, or whether the model just had a few more rounds to think the problem through.
To tell these apart, we set up five matched conditions on the same interface.
Direct answering. The system answers right after the worked examples, with no further exploration.
Without-tool answering. It goes through the same rounds, but receives no environment feedback.
Fixed-probe exploration. Instead of letting the system propose its own probes, we submit a randomized sequence of probes in their place.
Hindsight exploration. We hand back, unchanged, the probes from its own best trajectory; the system reads their results but no longer chooses what to test.
Autonomous exploration. The standard protocol: the system designs every experiment itself.
Held-out accuracy under the five conditions. Each dot is one system's accuracy at M4, and the line marks the median. From left to right, the conditions give the system more and more autonomy in exploration; shaded columns mark the conditions without environment feedback.
This shows that feedback is essential to exploration. Without-tool answering leaves AlienCode at 0.5–11.0%, and in AlienLogic three of the ten systems actually get worse.
Today's models lean more and more on longer chains of thought to raise their scores. But a few more rounds of reasoning only recycle the model's internal knowledge; exploration instead forces the model to interact with the outside world and obtain new evidence. Without real feedback, overthinking only makes a model hold more firmly to prior knowledge that was wrong in the first place. In one of our cases, a system that thought for four more rounds with no new evidence ended up abandoning rules it had correctly read off the worked examples.
Designing Your Own Experiments Helps Exploration More
Among the three conditions with feedback, the difference lies in who designs the probes. In AlienCode this matters enormously: the median falls from 67.4% under autonomous exploration to 41.2% under hindsight exploration, and then to 5.7% under fixed probes.
Hindsight exploration is the most telling of the three. It replays the probes of the system's own best trajectory, and the sandbox is deterministic, so it receives exactly the same evidence as autonomous exploration; it just no longer decides which probe to run next. Even so, autonomous exploration does better for nine of the ten systems, by a median of 17.4 points. Proposing its own hypotheses and running its own experiments works better than carrying out prescribed ones.
AlienLogic is different. There, autonomous and hindsight exploration are roughly even (a median difference of 0.2 points), but both sit far above fixed probes.
Discovering a Rule Doesn't Mean Using It Well
ExplorationBench also exposed a problem that is easy to overlook. Discovering a rule doesn't mean being able to use it.
To separate the two, we added open-book answering, which writes the complete rule set straight into the context, once before exploration and once after it. This gives a clean comparison. If the score from autonomous exploration falls far behind the open-book score, the bottleneck is discovering the rules; if autonomous exploration matches or beats it, the bottleneck is using them.
How the two worlds differ. Held-out accuracy at M4 under open-book answering before exploration (hatched), autonomous exploration alone (solid), and open-book answering after autonomous exploration (outlined).
The two sandboxes point in opposite directions.
In AlienLogic, simply being told the rules yields 96–100%, and no system's best trajectory matches that. Once the rules are revealed, the models complete the proofs almost perfectly, so the bottleneck is discovery.
AlienCode runs the other way. For seven of the ten systems, the best trajectory beats open-book answering before exploration. Under autonomous exploration GPT-5.6 Sol reaches 88.1%, against 69.5% when simply told the rules, and Hy4 preview scores 63.3% against 28.6%. In 26 of all 30 trajectories, being told the rules after exploring scores higher than being told them at the start, by a median of 14.5 points. Besides letting the model discover the rules, exploration also trained its ability to use them.
Most of AlienCode hinges on two off-by-one rules, the index shift of PLUCK and the slice shift of CARVE. Accuracy jumped by 30 points or more after a single round 13 times, and 12 of those came in exactly the round in which the model newly worked out the CARVE shift. In one trajectory, GPT-5.6 Sol used a single tool call to read four positions and one slice of a known list, and every value that came back differed from the manual's prediction. After that round, its rule report went from 8 correct rules to 28, and its held-out accuracy rose from 11.0% to 81.9%.
Accuracy jumps when the keystone rules are found. Each block is one system and each row one of its AlienCode trajectories. Triangles mark the milestone at which the rule report first states each off-by-one rule correctly.
Stating a rule correctly doesn't guarantee getting the task right. Even when a system's final rule report correctly states every rule a task needs, the task is solved only 73.4% of the time. Two of Kimi K3's trajectories show this. They explored under the same budget, and both ended up stating the index shift correctly. One finished at 79.0%. The other finished at only 5.7%, because inside function bodies it kept encoding numbers with a rule that holds only at the top level, so its indices went out of range.
Knowing a rule is not being able to use it. For each system, accuracy on tasks whose required rules the final rule report all states correctly (filled), and on tasks with at least one required rule missing or wrong (open).
Being able to state a rule is still some distance from using it correctly inside a program that combines many pieces.
What Else Does a Single Score Hide?
First, a single score hides how unstable exploration is. Answering the same tasks again at the same milestone barely moves the score, and the standard deviation across three answering passes is at most 4.7 points in AlienCode. But if the system explores again, the score can change completely. Under the same budget, Kimi K3's AlienCode endpoints range from 5.7% to 79.0%, and Gemini 3.8 Flash's from 6.2% to 79.0%. With results this spread out, the choice of aggregate changes the ranking: Qwen3.8-Max ranks sixth by Best@3 but third by the mean over its three trajectories.
Three parallel exploration trajectories end far apart, while repeated answering within each barely varies. Dots are each system's three trajectories at M4, the band spans their range, and the grey halo shows the answering noise of each trajectory.
The spread doesn't come from how much effort went in, since every AlienCode trajectory used 45 to 48 of its 48 tool calls. Yet with nearly the same number of tool calls, the best trajectories of different systems spent anywhere from 0.34 million to 11.6 million exploration tokens.
A single score also hides how far an ability carries. Move to another world and the ranking can change entirely. A system's rank in one sandbox barely predicts its rank in the other (Spearman correlation 0.35). Grok 4.6 ranks fifth in AlienCode and first in AlienLogic, while Gemini 3.8 Flash ties for third in AlienCode yet comes last in AlienLogic.
Progress also comes in leaps. In each system's best AlienCode trajectory, the largest single step accounts for 47–92% of the total gain, and the round in which it happens varies by system, anywhere from the first to the last. Further exploration can also erode earlier progress. In 6 of 30 AlienCode trajectories and 1 of 30 AlienLogic trajectories, the endpoint is at least 3 points below some earlier milestone. In one AlienLogic trajectory, Gemini 3.8 Flash correctly refused an unprovable theorem after the first round, then talked itself into a workaround that the checker rejects, and wrote that workaround into its rule report as a rule.
So looking only at the final score is an easy way to misjudge a system. Two systems can reach the same score in round four, one having climbed steadily step by step and the other having shot up and then fallen back. If you watch only the endpoint, you can't tell these two kinds of exploration apart.
What Should We Evaluate in the Future?
Put the three projects side by side, and you can see the handrails that evaluators built for models being taken away one after another:
In EvaLearn, the model learns from its experience on earlier tasks. In CL-bench, it learns from a complete context that humans have organized for it. In ExplorationBench, it has to find its own footing in an unfamiliar world. All three are after the same question: once its parameters stop updating and it faces rules it has never seen, can a model turn the evidence it gathers through interaction into progress that still holds up on the next task?
ExplorationBench isn't the end point either. The learning it measures happens only within the context of a single session, and once the session resets, everything the model worked so hard to learn is gone. What we want to look at next is the integration and consolidation of knowledge. Can an agent pick out the experience worth keeping from a large pile of exploration records and settle it into long-term memory, reusable tools, or even its parameters? And when the rules of the environment change, can it drop judgments that have gone out of date on its own?
These questions we leave to everyone working to make AGI a reality!
Acknowledgments
I would like to take this opportunity to add a few personal thanks. Special thanks to Maxm Pan for his guidance and help on ExplorationBench; I hope we will explore frontier evaluation techniques together and help advance the Hy models. Thanks also to Zhenghao Xiang for his high-quality work on building the benchmark and designing its rules; it has been a real pleasure working with him. I also thank Peizhong Gao for his help with the experiment design and analysis, data visualization and the academic writing, and my advisors, Tao Gui, Qi Zhang, and Xuanjing Huang, for their guidance and help. Finally, I am very grateful to Tencent Project UP for its strong support, which has let me devote myself fully to the frontier research I care about.
— Ming Zhang
Happy Mid-Autumn Festival, everyone! September 25, 2026
