A robot writes the physics its simulator is missing, then infers it from a few noisy experiments
A close read of EMPIRIC's residual world models: how a coding agent writes the physics a simulator lacks into a Python program, how Bayesian inference pins down those mechanisms' hidden parameters and state, and where the DreamCoder lineage's recognition model and library learning are still absent.
On 28 September 2026, Yichao Liang and fifteen co-authors posted EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning, a preprint from a group spanning Basis Research Institute, MIT, Cambridge, Princeton and Cornell that includes Joshua Tenenbaum, Tom Silver, Zenna Tavares and Kevin Ellis. The paper hands a robot a rigid-body physics engine and lets it write, in Python, the mechanics the engine is missing — the glue that cures between two blocks, the airflow from a fan, the heat under a jug — and then infer those mechanisms' hidden parameters from noisy observations. 1
Across five simulated tabletop domains the agent solves every one of its 25 runs, and the strongest baseline solves 16. On a physical robot it learns a fan's wind force and two blocks' masses from two gusts, then plans a two-block cascade that lands a block in a target patch the fan alone could not reach. 2
The result matters for a specific reason: the authors are the lineage. DreamCoder learned a library of reusable abstractions through wake-sleep Bayesian program induction, and the later systems in the line — LAPS, Stitch, LILO — each kept some of its parts. EMPIRIC keeps the Bayesian part and leaves the library out, which turns it into a clean measurement of how far program induction reaches when the programs are world models, the prior is a physics engine, and the role a recognizer normally plays is filled by a general-purpose coding model.
What the paper changes
The representation is a residual world model: a physics engine plus a code extension. The base simulator holds the scene's geometry and simulates rigid-body dynamics, with physical parameters such as mass and friction left exposed. The agent writes a Python program that subclasses the base engine, keeps its physics, and adds the mechanisms it lacks — forces, constraints, and quantities such as water volume, temperature, or a glued joint's cure progress. The authors use the word residual to name an executable extension: it can impose a constraint the engine never had, which an additive correction could not express. 2
A glue program shows the shape. It tracks a quantity the engine has no slot for — how long two blocks have stood in contact with glue on their faces — accumulates that contact time as recurrent state, and fuses the pair into one rigid body once the count passes a threshold. The threshold is never observed directly, so the program carries it as a parameter for the agent to infer. 2
Learning alternates between writing the program, inferring its parameters, and testing it through interaction. Four operations carry the loop. 2
- Write or revise the program and its predicates. The agent edits the program and a set of predicate classifiers — Python functions over observed features such as
Attached(a, b)orSeatedOn(s, l). Predicates check whether a skill did what the agent expected, hold a plan until a condition is met, and serve as subgoals for experiments. - Infer a belief over parameters and state. From its replay buffer and the current episode, the agent approximates a posterior over the program's parameters and the current model state: a mean-field distribution over parameters, a posterior over the base state such as object poses, and a point mass for the hidden recurrent state that each parameter draw implies.
- Plan and design experiments. EMPIRIC rehearses candidate skill sequences from joint draws of that belief and picks the sequence with the highest simulated success rate, or runs the experiment whose outcome carries the most mutual information about the parameters.
- Monitor and revise. After each skill the agent checks its expected predicates. A subgoal that holds on fewer than half the draws stops the plan and returns control to the agent, which can update the belief, replan, or edit the program.
What the evidence measures
Seven agents run over five simulated PyBullet domains and one physical task. Each domain hides a different piece of physics:
- Domino — lateral friction, with noisy poses;
- Bridge — glue deposition and bond curing, with cure progress hidden;
- Balloons — balloon lift that fades with height, and ceiling bursts;
- Boil — water filling and heating, with noisy volume and temperature;
- Fan — airflow and drag;
- Real robot — a fan's wind force and two blocks' masses.
A run contains one training task and one test task, two in Balloons, and counts only when every task is solved. Observations are noisy, with Gaussian position noise from 5 to 12.5 mm depending on the domain, and each run gets 10,000 to 20,000 environment steps across five seeds; simulation in the agent's sandbox costs no steps. Reward is evaluated on the whole trajectory, so a domino cascade that reaches the target the wrong way fails. 2
| Agent | Runs solved (of 25) | What it is given |
|---|---|---|
| EMPIRIC | 25 | Base engine, noisy observations, a persistent sandbox |
| Oracle dynamics | 25 | Ground-truth dynamics and parameters, still noisy observations |
| Standalone sim. | 16 | Writes its own transition model, with no base engine |
| Direct agent | 16 | No simulator; acts from observations and execution feedback |
| Direct + scene | 14 | Scene assets and engine code, which it may simulate with |
Table from the paper's results; all seven agents use Claude Opus 5 at high effort. 2
EMPIRIC's 25 compare with 16, 16 and 14, and the pooled differences are significant by a two-sided Fisher's exact test at p ≤ 0.002. Oracle dynamics also solves 25, which sets the ceiling on success, and the step comparison there is more interesting: EMPIRIC uses 12–18% more steps than Oracle in Domino, Bridge and Balloons and 74% more in Boil, yet 8% fewer in Fan. Correct dynamics on their own buy success; efficiency still has to be earned, because Oracle estimates object poses from noisy readings and chooses its own actions. 2
Two ablations strip out parts of the machinery. Removing the fitted parameter belief drops success to 20 of 25 runs; removing explicit uncertainty support drops it to 21. Both ablations still solve all five Boil and Fan runs, and on those two domains they sometimes spend fewer steps than EMPIRIC — 988 against 1,331 in Boil, 1,851 against 2,475 in Fan. The paper reads that reversal as task structure: a near-constant filling rate, a conservative fill target and opposing-fan braking all make those tasks insensitive to the quantities being estimated, so inference on them buys nothing. 2
Balloons separates the ablations from the full system, and it does so through irreversibility. EMPIRIC solves 5 of 5 runs there, where the two ablations solve 2 and 3, and every one of the five ablation failures ends with a balloon bursting on the ceiling. A release cannot be undone, and a forward simulation that predicts 15 mm of ceiling clearance can be wrong by more than that margin, so the agent has to price its own uncertainty before letting go. 2
Where the gain comes from
The interaction savings are bought with computation. The median EMPIRIC run uses 424 simulator rollouts and 162 language-model turns, against 115 turns for the Direct agent, and its median recorded run takes 107 minutes against 64. Environment steps are the currency the benchmark charges, and the sandbox is free on that clock; wall time is where the cost moves. 2
The base engine is where the efficiency comes from. In Fan, the agent adds a single directional force and lets the engine supply contact, gravity and every robot skill, so one learned force extrapolates to arrangements the agent never saw. In Domino it adds no mechanism at all and fits only the engine's friction, raising it from 0.10 to 0.50, after which the model turns a straight cascade through 90°. A model built this way can be wrong in detail and still do work: in Bridge EMPIRIC mispredicts the lifted beam's pose, uses the model anyway to screen individual placements, and then measures the assembled beam's tilt before choosing its final release height. 2
The real-robot run shows the same division of labour with two observations. The agent blows at the green block first, where its belief disagrees most about the outcome, finds that only the gray block moves, and from two gusts learns a wind force that decays after the fan stops and infers both masses. At test its model predicts that no placement of the green block alone reaches the moved patch, so the agent stands the gray block upwind and knocks the green one in. The predicted slide of 10.1 ± 2.3 cm matches the measured 9.9 cm. 2
Position in the DreamCoder lineage
| Method | Persistent object | Neural search guidance | Abstraction engine | Substrate | Scaling bottleneck |
|---|---|---|---|---|---|
| EMPIRIC | Base physics engine plus agent-written residual programs, with a belief over their parameters and hidden state 2 | A frozen coding LLM writes and revises the mechanism program and predicates; no recognizer is trained on the agent's own solves | None across domains; mechanisms are written per domain | Tabletop manipulation in PyBullet, plus one physical robot | Scene geometry and object features are supplied; simulation and language turns |
| DreamCoder | Typed λ-calculus DSL plus a recognition model trained by wake-sleep 3 | Recognition model trained on replayed and imagined tasks | Bayesian refactoring extends the language with new abstractions | Synthetic symbolic domains, graphics | Enumerative search explosion |
| LAPS | DreamCoder library plus a search model conditioned on language 4 | Neurally guided search trained jointly with library learning | Library learning driven by natural-language annotations | String editing, image composition, scene reasoning | Needs natural-language annotations |
| Stitch | Compressed corpus of programs plus learned abstractions, symbolic throughout 5 | Top-down search guided by the corpus | Anti-unification over e-graphs | Large DSL corpora | Memory on massive corpora |
| LILO | Named, documented code library 6 | LLM synthesis prompted with library functions and specifications | Stitch compression plus automatic documentation | String editing, scene reasoning, graphics composition | Compression limits; proposal fidelity |
Comparison compiled by this article from the cited papers.
Three lineage slots are filled here, and something outside the system fills each one. The representation — the base engine and the extension interface — is written by the paper's authors. The search guidance comes from a frozen coding model that never updates on the benchmark. The prior over which mechanism to try is the model's pretraining, and nothing flows back from EMPIRIC's own solves into that prior.
The compounding parts are missing. No recognizer is trained on the agent's solutions, so a domain solved once leaves the proposal model exactly as it was, and no library of mechanisms grows across the five domains: the residual program for Bridge stays in Bridge, and Fan begins from the same base engine as Domino. What does persist is within a run, where the sandbox and the current program carry from the training tasks to the test task, which is the reuse the benchmark rewards with fewer steps.
Verbatim quotes
"A robot should be able to learn through experiments how unfamiliar objects behave and interact, then plan with that knowledge." — the authors, opening the abstract. 2
"The learned programs can introduce new forces, constraints, and hidden state, and Bayesian inference estimates their parameters and states from noisy observations." — the same authors. 2
"A learned model can be useful without being accurate everywhere: in Bridge, the model mispredicts the lifted beam's pose, yet the agent uses it to screen individual placements and then measures the assembled beam's tilt to choose its final release height." — the same authors, on the comparison with given dynamics. 2
"Interaction savings come at a computational cost: the median EMPIRIC run uses 424 simulator rollouts and 162 language-model turns, compared with 115 turns for Direct agent, and its median recorded runtime is 107 minutes, against 64 for Direct agent." — the same authors, in the limitations. 2
"In our experiments, the agent is given the scene geometry, although object poses are noisy and mechanism state is hidden" — the same authors, naming what the setup assumes. 2
The unresolved test
EMPIRIC shows that a physics engine plus a frozen coding model plus Bayesian inference goes a long way on tabletop manipulation, and it also marks where the lineage's compounding claim would have to show up. The one experiment that would settle the question keeps the five domains and changes one thing: after each domain, add the residual program and its predicates to a shared library, and let the proposal model — the same LLM, or one fine-tuned on the deposited mechanisms — propose the next domain's mechanisms conditioned on that library. Then re-run the later domains and measure steps to solve against the frozen system as the control.
If conditioning on the deposited mechanisms bends the step curve down, the per-domain programs were standing in for the library and the trained recognizer the lineage builds, and EMPIRIC's ceiling is a missing component rather than a property of the problem. If the curve holds, a hand-written engine with a frozen proposer is a real alternative to a grown one on this class of tasks, and the lineage's compounding bet meets a benchmark where compounding would buy little.
References
- 1
- 2
- 3
- 4
- 5
- 6
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.