Baba in Wonderland: Online Self-Supervised Dynamics Discovery for Executable World Models
Quick Answer
The paper presents Alice, a closed-loop system that enhances online executable world-model learning by refining state-dependent dynamics under prior misalignment.
Quick Take
The paper presents Alice, a closed-loop system that enhances online executable world-model learning by refining state-dependent dynamics under prior misalignment. By treating failed updates as structural signals, Alice improves hypothesis generation and exploration in environments like Baba in Wonderland, significantly advancing the learning process without relying on rule descriptions or reward signals.
Key Points
- Alice improves hypothesis generation by refining conflicts from failed candidate updates.
- The system enhances exploration toward novel transitions in executable world models.
- Experiments show significant improvements in learning under prior misalignment.
- Alice operates without rule descriptions, reward signals, or reliable lexical priors.
- The approach is validated in a variant of Baba Is You, preserving simulator dynamics.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Executable world models can be read, edited, executed, and reused for planning, but only if the program captures the environment's transition law rather than semantic shortcuts in its surface vocabulary. We study online executable world-model learning under prior misalignment, where an agent must induce state-dependent dynamics from interaction evidence alone, without rule descriptions, reward signals, or trustworthy lexical priors. We introduce Alice, a closed-loop system that treats failed candidate updates as structural signal: when a candidate explains a new transition but loses previously explained ones, the preservation conflict reveals dynamics that the current program had conflated. Alice refines these conflicts into hypothesis classes that both provide compact, class-stratified preservation counterexamples for update and guide frontier exploration toward transitions that are novel and underrepresented with respect to the current program. We evaluate Alice on Baba in Wonderland, a prior-misaligned variant of Baba Is You that preserves simulator dynamics while replacing semantically meaningful rule-property labels with unrelated words. Experiments show that Alice substantially improves executable world-model learning under prior misalignment, and ablations show that both class refinement and class-aware exploration contribute.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.16725 [cs.AI] |
| (or arXiv:2605.16725v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.16725 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: HyeongYeop Kang [view email]
[v1]
Sat, 16 May 2026 00:18:22 UTC (2,298 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Procedural Memory Distillation: Online Reflection for Self-Improving Language Models
Procedural Memory Distillation (PMD) enhances reinforcement learning by converting cross-episode signals into reusable memory, improving Qwen3-8B and OLMo3-Instruct-7B models by 3.8-5.5% on SCIKNOWEVAL and 7.9-13.6% on . The co-evolution of policy and memory allows for more effective self-supervision, demonstrating significant performance gains when both components are active.