Certified pattern edits train a search ranker: 63 of 80 cases, and 0 of 57 with an unseen action
A close read of SemDraft: how three certification checks turn recovered pattern-editing programs into supervision for a learned search ranker, why its edge over the authors' heuristic only appears once the beam widens, and what the 0-of-57 held-out-composition result says about the DreamCoder lineage's recognition model.
On 15 October 2026, Zhuotao Xu, Chi Zeng, Liying Yang, Xinxing Yu and Yanyan Liang, all at Macau University of Science and Technology, posted Learning to Search for Executable Patternmaking from Certified Programs. 1
Their system, SemDraft, is given two sewing patterns: a base and an edited target. It recovers a program of patternmaking edits — lengthen a bodice, move a dart, swap a sleeve — that turns the first pattern into the second. 2
Garment archives keep the edited patterns and none of the steps. SemDraft searches for the steps, then puts each recovered program through three checks before accepting it. The accepted programs train a small transformer to rank candidate edits inside the same search. 2
At beam width four, the trained ranker certifies 63 of 80 order-sensitive test cases. The authors' own ranking heuristic certifies 51, a lookup of action-type statistics 53, and a random order 37.7. 2
Two quieter numbers sit in the same table. With a greedy beam of one, the trained ranker certifies 35 cases and the heuristic 51. On a garment family held out of training, the ranker certifies none of that family's hardest cases, where the random order certifies some. 2
The part that matters here is which slot of the DreamCoder lineage SemDraft fills. DreamCoder's recognition model is a network that guesses which programs would explain a task and uses those guesses to order the search; a wake-sleep loop trains it on replayed and imagined problems. 3 Recent neurosymbolic work has mostly parked a frozen language model in that slot. SemDraft trains the network on the system's own accepted programs, and then measures where the training stops helping.
What the paper changes
A pattern state holds panels, seams, components and darts, each with labelled edges and millimetre parameters. Edits come from nine semantic actions: introduce an internal dart, transfer one, redistribute ease, true a seam pair, adjust a component's length, apply an axis slash-and-spread, insert a style line, and attach or detach a template component such as a sleeve. 2
A deterministic compiler lowers those actions to geometric primitives and rejects any transition that breaks closure, self-intersects, or violates seam and dart validity. One action changes several panels, their seam partners and their symmetry groups at once. 2
Determinism is what makes a recovered program checkable. The same program always reproduces the same state and the same artefact hash, so three checks can be run against it. 2
The first check recompiles the whole program from the base pattern and compares hashes. The second classifies every pair of actions as ordered or interchangeable, by compiling both orders from a shared prefix and comparing the results. The third replays the compiled program in an evaluator kept apart from the search that found it. A minimality rule then rejects a program whose proper sub-program already reaches the target. 2
The certificate is worth more than a pass mark. It hands back the program, its dependency DAG, and, for every intermediate state that can be materialised, the set of actions that are valid next. That set is the supervision. 2
The ranker is a four-layer transformer with 128-dimensional embeddings and four attention heads. Panels, target panels, components, darts, past actions and a global token become inputs; separate heads score the action type (ten classes including STOP), point at an anchor panel or component, and regress amounts and slots. Training maximises the summed probability of every valid next action at every state, and takes under five minutes on one GPU. 2
The comparison the paper cares about holds everything else fixed: the same nine proposal providers, the same beam, the same stopping rule, the same certifier, the same budget in engine executions. Only the ranking signal changes. 2
What the evidence measures
Every corpus comes from one generator. A case starts from a neutral base pattern in one of four garment families — tee, skirt, pants, dress — and applies a hidden edit program of depth two to three (Tier A), four to five (Tier B) or six to eight (Tier C). Every Tier-C program contains a template swap. 2
The authors built a 300-case development corpus, then a second 300-case corpus whose configuration digests were recorded and made read-only before anything ran on it. Splits were declared in advance: 24 calibration cases, an 80-case Tier-C main test, and a 20-case easy reference. A replication draw of 80 further Tier-C cases and two 20-case shift sets were added later, each frozen before its first run. 2
The search that produces the training data certifies 77, 70 and 57 of the 80 held-out cases in Tiers A, B and C. A Tier-C search costs a median of 1,442 engine executions and 1,315 seconds of wall clock. Those runs yield 1,510 certified terminals over 185 cases, and 2,846 distinct intermediate states after de-duplication. 2
| Ranker | Certified at beam 1 | Certified at beam 4 | Median search executions, beam 4 | Median wall clock, beam 4 |
|---|---|---|---|---|
| Random order (mean of three seeds) | 17.7 | 37.7 | 178 | 61 s |
| Template lookup | 27 | 53 | 176 | 60 s |
| Authors' heuristic | 51 | 51 | 351 | 349 s |
| Learned prior, linear imitation | 35 | 61 | 176 | 61 s |
| Learned prior, DAG supervision (mean of two seeds) | 35 | 63 | 180 | 61 s |
Certified cases out of the 80 Tier-C main-test cases, at the 5% budget of 1,000 search executions. Every row shares the candidates, beam, stopping rule, certifier and budget; only the ranking changes. 2
Run unchanged with its full wall-clock budget on the same 80 cases, the teacher's reference configuration certifies 30, at a median of 1,320 executions and 1,373 seconds per case. The 50 cases it misses become the paper's hardest comparison. 2
Before any search, the ranker can simply be asked for the next action. Teacher-forced on a held-out split, it names a valid action type 91.9% of the time against a 48.8% majority baseline, names the right anchor entity 99.0% of the time, and halves the error of a mean predictor on millimetre amounts. 2
Where the gain comes from
Most of the 53 certified cases the template lookup reaches need no geometry at all. It orders candidates by the empirical probability of an action type given the tier and the step index, and on this corpus that is a strong signal, because each tier fixes its action-type order. 2
The remaining ten cases are where the geometry has to decide, and the learned ranker needs room to use it. Narrow the beam to one and the ranker falls to 35 while the heuristic holds at 51; the heuristic compiles every candidate and ranks the compiled states, so it corrects a wrong first choice as it goes. 2
Widening the beam pays across the board: the ranker goes from 35 to 63, the lookup from 27 to 53, the random order from 17.7 to 37.7, and the heuristic stays at 51. 2
The two beams also hold different things. On the same cases, the heuristic's beam carries a median of 12 states that are reorderings of the same actions, and the ranker's carries 2. After the main results were read, the authors added a merge of duplicate beam slots and pre-registered its check; the ranker rises from 63 to 66 and the heuristic falls from 51 to 43. 2
The gain concentrates at depth seven, where the ranker certifies all seven tee and all seven dress cases and the heuristic one of each. At depth eight the two are level, at 10 of 24. Pants is the family the heuristic owns, with 18 of 20 cases against the ranker's 11. 2
Cost runs the same way. On the 44 cases both signals certify at beam four, the ranker needs a median of 160 search executions and the heuristic 325, because the heuristic pays for its ranking by compiling every candidate at every step. 2
Where the training stops carrying
Train the ranker on three garment families and test it on the fourth, and the gains reverse. It beats the random control only on skirts, loses to it on tees, pants and dresses, and certifies none of the held-out family's Tier-C cases, where the random order sometimes stumbles onto a correct order. 2
Holding out composition is harsher. Train on Tiers A and B, so the ranker never sees a template swap, and it certifies 0 of 57 Tier-C cases against 20 for the random ranker, because it gives the swap negligible probability. 2
The two distribution shifts land in between. Unseen parameter values drop the ranker from 79% certified to 65%, and changed templates to 70%, against 50–55% for the heuristic and 45–50% for the lookup. Its edge over the heuristic is two cases on the parameter shift and four on the template shift, and both bootstrap ranges include zero. 2
The 15-point edge over the heuristic comes with a wide interval. Resampling the twelve family-by-depth cells instead of the cases widens it to −10 to 40 points on the main test. 2
Certification also costs the ranker cases. Pairs that never become adjacent keep their recovered order without an executed test, and the minimality rule rejects eight of its tight endpoints at beam four, every one of them a program that reaches the target with one redundant ease step. 2
The corpora are templated. Hidden programs follow a per-tier and per-depth action order and draw their parameters from small value sets, which is how a lookup of action-type statistics reaches 53 of 80 without looking at a single contour. 2
The authors state the boundary themselves: controlled synthetic identifiability benchmarks from one generator, polygonal geometry, the same nine-action ontology the search uses. They claim no transfer to another generator, and they note that a certificate shows executability and order relations against their compiler rather than how a professional would draft. 2
Position in the DreamCoder lineage
The table lines SemDraft up against the four systems this channel tracks. The comparison is compiled by this article from the cited papers; the SemDraft row is the paper's own description of itself.
| Method | Persistent object | Neural search guidance | Abstraction engine | Substrate | Scaling bottleneck |
|---|---|---|---|---|---|
| SemDraft | Trained ranker weights; certified programs stay a training corpus 2 | Transformer trained once on the system's own certified programs, ranking candidates before compilation | None: the nine-action vocabulary is fixed by hand | Synthetic sewing-pattern editing, four garment families | A ranker fitted to one generator's action distribution |
| DreamCoder | Typed λ-calculus DSL plus a recognition model trained by wake-sleep 3 | Recognition model trained on replayed and imagined tasks | Bayesian refactoring extends the language with new abstractions | Synthetic symbolic domains, graphics | Enumerative search explosion |
| LAPS | DreamCoder's library plus a search model conditioned on language 4 | Neurally guided search trained jointly with library learning | Library learning driven by natural-language annotations | String editing, image composition, scene reasoning | Needs natural-language annotations |
| Stitch | Compressed corpus of programs plus learned abstractions 5 | Top-down search guided by the corpus | Anti-unification over e-graphs | Large DSL corpora | Memory on massive corpora |
| LILO | Named, documented code library 6 | LLM synthesis prompted with library functions | Stitch compression plus automatic documentation | String editing, scene reasoning, graphics composition | Compression limits; proposal fidelity |
SemDraft fills the recognition slot and leaves the library slot empty. The nine actions are chosen by hand once, the compiler never grows a new operator, and nothing about a solved case changes the vocabulary for the next one. 2
Against DreamCoder's supervision, SemDraft's is more precise. A wake-sleep replay tells the recognition model which programs solved a task; a SemDraft certificate tells the ranker which of several next actions are valid at each state, and which pairs of actions may be reordered. That is a richer target for the same slot. 3
Where it falls short, it falls short on what the lineage does with that slot. DreamCoder retrains the recognition model each time the language grows, so the network learns to search a distribution that changed. SemDraft trains once, against a fixed vocabulary, on one generator's tasks, and its held-out-composition split shows what that costs. 2
There is also no sleep. The ranker learns from problems the search already solved; it never practises on a task the teacher failed, which is where a wake-sleep loop would spend most of its imagined programs. 3
Verbatim quotes
"Our insight is that a deterministic compiler makes a recovered sequence checkable." — the authors, on the move the whole method rests on. 2
"A certificate is the unit of supervision: it yields the program, its dependency DAG and, for every materialised intermediate state, the set of valid next actions (nodes whose required predecessors are complete) and whether stopping is allowed there." — the same authors, naming what the ranker is trained on. 2
"The teacher ranks a candidate only after compiling it; the prior ranks candidates before compilation." — the same authors, on why the two signals cost different amounts. 2
"When every case containing a swap is held out, the policy solves 0 of 57 Tier-C cases against 20 for the random ranker: it has never seen a template swap and assigns it negligible probability." — the same authors, in the appendix. 2
"The development corpus and the frozen evaluation set are controlled, synthetic identifiability benchmarks, not evidence of industrial generalisation." — the same authors, in their limitations. 2
The unresolved test
The held-out-composition split is the place to test what the lineage actually claims. A recognizer that is retrained whenever the language grows should be able to search a distribution it has never met, because the vocabulary carries the structure across.
An experiment that would settle it keeps the rest of the pipeline fixed and adds the missing slot. Mine recurring action subsequences out of the certified programs, admit the useful ones as named composite actions through the same certification gate, and let the compiler treat each as one step. Retrain the ranker on the enlarged vocabulary and re-run the 57 swap-bearing cases.
If those cases start certifying, the zero was a vocabulary failure, and the recognition model needed the DSL to grow beside it. If they stay at zero, the ranker is failing on the geometry of unseen states, and a larger vocabulary will not reach them. Either answer is worth having: the first makes library growth a prerequisite for the recognizer, and the second puts a limit on how far either mechanism carries a search in a new domain.
References
- 1
- 2
- 3
- 4
- 5
- 6
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.