Dozens of banned expression schemas and a laptop enumerator beat a coding agent on 14 continuous games
A close read of agentic sketching: how one DSL and a hand-written ban list shrank the search space enough for plain enumeration to outscore a Codex agent on 14 continuous games, and where the combination and a PPO baseline each won.
On 21 September 2026, Maxim Gumin and Daniel Ritchie of Brown University, Hsueh-Ti Derek Liu of Roblox, and Victor Zordan of Clemson University posted Synthesizing Reactive Character Behaviors for Continuous Games via Programmatic Policy Search, a paper to be presented at SIGGRAPH Asia 2026. It searches over a domain-specific language for reactive game policies instead of fitting a neural controller, and it runs three versions of that search over the same 14 continuous games under the same 20-minute budget per game. 1
Normalized so that exhaustive symbolic enumeration scores 1.00, an off-the-shelf coding agent given the same sandbox scores 0.76, and the two combined score 1.23. 2 The enumeration that beat the agent ran on one laptop CPU with all cores busy, scoring whole DSL expressions in the simulator, while the agent ran on OpenAI Codex with GPT-5.5 at extra-high reasoning. 2
The ordering measures a choice the DreamCoder lineage also faces, between handing a search structure and letting it find structure, and it puts the search above the learned proposer before putting the combination above both. The paper's design decisions then show where a program search's leverage sits when the designer owns the language: in the grammar, in a hand-written list of banned expression forms, and in a loop that fills one hole at a time. The system publishes each game's policy as a finished program, and the abstractions it finds inside one game stay inside it. The comparison table further down, which places this system against DreamCoder, LAPS, Stitch and LILO, is this article's own reading; the paper keeps its claims to policy search.
What the paper changes
The language comes first. One DSL covers all 14 games, written in a Lisp-like S-expression syntax over a small set of types, with sensors built on raycasting: a policy asks how far a ray travels before hitting something, which is a common primitive in game engines and robot simulators. Low-dimensional sensors matter for search, because few distinct values can appear inside a program. Higher-order functions do more of that work:
(maxDirection (raydist #0)) returns the direction in which a ray travels farthest, letting a compact expression name a whole class of geometric decisions. A policy outputs a target direction on the unit circle plus two booleans, forward and use, and the character has a finite turn rate, so movement has bounded curvature and the policy has to reason about facing rather than pick a velocity. Pursuit, kiting and obstacle avoidance all fall out of that. 2Second comes the pruning. Bottom-up enumeration builds expressions from the grammar and tests complete candidates in the simulator, and its cost grows doubly exponentially with depth. Policy search generates the states its policies meet, so the usual tool of observational equivalence, which compares expressions on a fixed example set, has to be replaced. The authors write antipatterns instead: expression schemas banned outright during enumeration. Some are exact redundancies, such as
(not (not A)), (if (not B) A C) where (if B C A) says the same thing, (left (right A)), or (maxDirection (alignment #0 V)), which returns V itself. Some are context-sensitive, excluding a nested test of a condition inside a branch that already tested it. Others are deliberate trades: (distance (equipped A) B) is banned because measuring between base objects is usually enough in these domains. The gain compounds locally, because discarding one expression discards every larger expression that would have used it. 2Third comes the split between agent and enumerator, which the authors call agentic sketching. The coding agent writes a sketch: a DSL program containing one typed hole, a maximum size for the expression that will fill it, and optionally antipatterns specific to that hole's context. It then calls the enumerator as a command-line tool, and the tool enumerates every well-typed expression of the hole's type up to the bound, substitutes each into the sketch, simulates the completed policy, and returns the best-scoring subexpressions at each size with their scores. The agent decides from that feedback whether to keep the sketch, revise the surrounding structure, raise the bound or try a different decomposition. Across the benchmark the agents made 213 search attempts, 203 of them valid, reached 150 distinct sketches, and raised a bound on 46 attempts, 21 of those a consecutive revisit of the same sketch. 2
What is new relative to earlier programmatic policy search is the shared language. Prior evaluations concentrated on grid worlds such as Karel and MiniGrid, or on a single continuous task with its own policy language, as in racing and parking. Here one policy language spans 14 continuous games, which is what makes an enumerator and a language model comparable on the same search space. The authors are explicit that the individual behaviors resemble ones in Unity ML-Agents and in multi-agent testbeds such as MPE and Google Research Football; the contribution is that all 14 share one policy language. 2 The benchmark and algorithms are released in the BeyondTheGrid repository.
What the evidence measures
Each method gets a 20-minute wall-clock budget per game, then the best policy found so far is frozen and rolled out. A policy's score is the minimum over 5 to 50 random seeds, 50 for single-episode games such as MouseCats and 5 for games that chain short subepisodes such as Pathfinding; levels are procedurally generated with unwinnable configurations ruled out; and a candidate that already trails the best-so-far minimum on one seed skips its remaining seeds. Minimum aggregation is what makes the budget comparable across methods, and it keeps evaluation cheap: the search methods averaged 1.07 rollouts per policy. 2
The per-game best scores, each value the minimum across that game's seeds:
| Game | Search | Coding agent | Agentic sketching | Sketching, local antipatterns removed | PPO |
|---|---|---|---|---|---|
| ButtonCats | 21 | 17 | 24 | 22 | 157 |
| ButtonCheese | 12 | 9 | 14 | 14 | 9 |
| ButtonCheeseCats | 7 | 6 | 12 | 9 | 6 |
| Chase | 6 | 2 | 8 | 8 | 35 |
| Dodgeball | 12 | 3 | 9 | 9 | 0 |
| Football | −1 | −1 | 0 | −2 | 0 |
| Hide | 1747 | 1675 | 1858 | 1812 | 581 |
| Lawnmower | 57 | 52 | 60 | 60 | 66 |
| MouseCats | 1088 | 758 | 1572 | 1266 | 595 |
| MouseCheeseCats | 11 | 6 | 15 | 11 | 14 |
| Pathfinding | 5 | 6 | 8 | 6 | 2 |
| ProtectTheChild | 20810 | 16402 | 22684 | 20810 | 11098 |
| Seek | 4 | 4 | 4 | 4 | 1 |
| Vacuum | 39 | 30 | 50 | 50 | 33 |
Table 2 from the paper, reorganized with games as rows. Every value is the minimum score over that game's random seeds; higher is better. 2
Read down the columns and the ordering holds case by case. The coding agent trails pure enumeration on eleven of the fourteen games, ties on Football and Seek, and leads on Pathfinding. Agentic sketching leads enumeration on twelve games, ties on Seek, and trails on Dodgeball, where pure search found a policy scoring 12 against the combination's 9.
PPO is the fourth column, and the paper trains it as a serious baseline: MaskablePPO from Stable-Baselines3, 100 million simulator frames per game, with hyperparameters tuned by successive halving, at an average of 140 minutes of wall clock per 10⁸ frames. It wins on average, and the paper traces that win to two games, where its agent learned to circle the ButtonCats button and press it rapidly and to collide repeatedly with the Chase target. Program search discovered the same ButtonCats behavior, and the DSL's single numeric constant, 2.5, kept it from pressing as often. Where reward is sparse or late, PPO does badly: Dodgeball pays only after a ball is picked up, thrown and hits a moving target, and ProtectTheChild concentrates the scoring difference near the end of each episode. Program search sidesteps that credit assignment problem by evaluating complete playouts, and its language names the ball and the target directly. 2
Where the gain comes from
| Method | Normalized score (search = 1.00) | Policies tested | Largest program searched |
|---|---|---|---|
| Coding agent | 0.76 | 539 | 25–163 |
| Search | 1.00 | 1.99 × 10⁶ | 10–30 |
| Agentic sketching | 1.23 | 1.71 × 10⁵ | Holes of 9–36 inside sketches of 27–111 |
Table 1 and Table 3 from the paper. Policy tests are averages across the benchmark; the size column gives the largest expression each method allowed itself, per game. 2
The combination reaches the highest score while testing an order of magnitude fewer policies than enumeration, and the accounting explains why: 91% of its time goes to the language model, 8.2% to testing policies, and 0.8% to enumerating expressions. 2 Its division of labour shows up in the sizes. Enumeration stops at expressions of 10 to 30 nodes, and the agent on its own writes programs of 25 to 163 nodes that still lose. Agentic sketching lets the agent build a large structure of 27 to 111 nodes while keeping every hole small enough, 9 to 36 nodes, for the enumerator to exhaust.
The ablation isolates the sketch-specific antipatterns, the ones the agent writes for the context of a particular hole. Removing them lowers the score on 8 of the 14 games and leaves the other 6 unchanged: Pathfinding falls from 8 to 6, MouseCats from 1572 to 1266, ButtonCheeseCats from 12 to 9, ProtectTheChild from 22684 to 20810, Hide from 1858 to 1812, ButtonCats from 24 to 22, MouseCheeseCats from 15 to 11, and Football from 0 to −2. 2
Two details say what the discovered policies are like. Before simulation, a static filter keeps only candidates that call a movement operator, and in games with an on-use trigger, only those that call use. The Pathfinding policy that survived navigates a maze using raycast distances, escaped narrow cul-de-sacs in the authors' tests, and failed in wider ones and in zigzag mazes that need a coordinated sequence of turns, a limit of its local heuristic rather than of the search. The paper reads the general pattern this way: enumeration is strong for cheap local search, while a coding agent is useful for proposing the larger program structures that make local search effective. 2
Position in the DreamCoder lineage
| Method | Persistent object | Neural search guidance | Abstraction engine | Substrate | Scaling bottleneck |
|---|---|---|---|---|---|
| Agentic sketching | Fixed hand-written DSL; each game's sketches and fragments end with the game | Pretrained coding agent writes a sketch with one hole; its weights come from outside the benchmark | Hand-written antipatterns remove banned expression schemas | Reactive policies in continuous games, scored by simulator rollouts | Budget split between model calls and enumeration; the DSL holds one numeric constant |
| DreamCoder | Typed λ-calculus DSL plus a recognition model trained by wake-sleep 3 | Recognition model trained on replayed and imagined tasks | Bayesian refactoring extends the language with new abstractions | Synthetic symbolic domains, graphics | Enumerative search explosion |
| LAPS | DreamCoder library plus a search model conditioned on language 4 | Neurally guided search trained jointly with library learning | Library learning driven by natural-language annotations | String editing, image composition, scene reasoning | Needs natural-language annotations; inherits the refactoring cost |
| Stitch | Compressed corpus of programs plus learned abstractions; symbolic throughout 5 | Top-down search guided by the corpus | Anti-unification over e-graphs, orders of magnitude faster than deductive refactoring | Large DSL corpora | Memory on massive corpora |
| LILO | Named, documented code library 6 | LLM synthesis prompted with library functions and specifications | Stitch compression plus automatic documentation of new abstractions | String editing, scene reasoning, graphics composition | Compression limits on arbitrary code; proposal fidelity |
Comparison compiled by this article from the cited papers.
Three of the lineage's slots are filled here, and each is filled by something an author wrote rather than something the system learned. The DSL is what makes a continuous behavior space enumerable, and it arrives frozen; the antipattern list is what removes redundant program forms, and every entry in it was typed by hand; the coding agent proposes the structure the enumerator then completes, and it arrives pretrained from outside, working from a prompt that describes the DSL and the game. Within a game the agent does learn from scores, since the enumerator's results feed its next sketch. Across games each search starts over from the same blank context, and the subexpressions that solved one game stay behind when the search moves to the next.
The paper's own future-work list names the two mechanisms the lineage automates. Antipatterns are currently hand-coded, and a natural next step is to derive them automatically by static analysis or abstract interpretation. And the coding agent could build simplified versions of a game, in the manner of curricula or unit tests, discover useful policy fragments there, and transfer those fragments back to the original task. 2
One limitation is stated plainly and bears on the lineage's premise that a better language is the compounding asset. The DSL contains a single numeric constant, 2.5, so thresholds, radii, gains and timing stay fixed across every game, which keeps enumeration small and rules out behaviors that depend on precise numbers. 2 A library that grows buys search; a language that fixes its parameter space buys comparability and pays in expressiveness.
Verbatim quotes
"Perhaps surprisingly, bottom-up enumeration outperforms the coding agent alone, despite running without an LLM on a single laptop CPU." — the authors, in the introduction. 2
"This result suggests that exhaustive symbolic search is a strong baseline when the DSL is compact and carefully designed." — the same authors. 2
"Such antipatterns trade a small amount of expressivity for a much smaller search space." — the same authors, on the design-choice antipatterns. 2
"We currently hand-code antipatterns. A natural next step is to derive them automatically using static analysis or abstract interpretation." — the same authors, listing their own missing piece. 2
"Despite testing an order of magnitude fewer policies than pure search, agentic sketching achieves the highest average normalized score." — the same authors. 2
The unresolved test
The result separates two ways of making a program search cheap, and the paper supplies the ingredients for telling them apart, leaving the comparison itself to be run. Here the design of the language plus a hand-written ban list did the work that, in the DreamCoder lineage, is done by a growing library and a recognition model trained on the system's own solves.
A test that would settle the difference: keep this benchmark fixed, mine the subexpressions that recur across the policies solved in the first games, add them to the DSL as new primitives, and replace the hand-coded antipatterns with rules derived from the solved programs. Then re-run the later games with the grown language and the derived rules, with the frozen system as the control, and score search budget against final score. If growth shifts the curve, the hand-designed DSL was standing in for library learning, and the compounding component of the lineage is what this benchmark was missing. If growth leaves the curve where it is, a designed search space stands as a real alternative to a grown one, and the lineage's compounding claim meets a benchmark where that growth would buy little.
References
- 1
- 2
- 3
- 4
- 5
- 6
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.