Lean × AI-for-Math

A 3-card audit of Pythagoras-Prover, a new open-source Lean 4 prover family: the 4B model reports 86.1% pass@32 on MiniF2F-Test, while the 32B model reports 93.0% and 93/672 on PutnamBench.

MiniMax M3 + MaxProof population-level test-time scaling reaches 35/42 on IMO 2025 and 36/42 on USAMO 2026, exceeding the gold-medal threshold on both. Claim audit: scores are expert-verified and confirmed — but proofs are natural language only; no Lean 4 kernel is involved. Verifier is the M3 model itself (generative LLM). Benchmark advance in the informal competition track, not the formal verification track.

TheoremBench (arXiv:2606.09450, Jun 8 2026) introduces a new Lean 4 benchmark built from ~100 classical theorems (Wiedijk list), expanded into 1,142 instances in two formats: plain-main and premised (with explicit supporting subtheorems). Key result: explicit premises produce a 5× pass@64 lift for DeepSeek-Prover-V2-7B (5.3% → 26.3%) but zero gain for the non-reasoning SFT baseline. New finding: provers write 8–16× more tokens than reference proofs — brute-force tactic traces, not compact proof plans. Lean 4 kernel verification throughout. Single-lab evaluation; no SOTA agentic systems (LEAP, Goedel-Architect full scale) tested.

A closed-source commercial tool (Aristotle) failed both Erdős–Graham and ESS #1196 after 40+ hours, leaving sorry placeholders. LeanMarathon — a new open-source multi-agent harness from Warwick/Michigan/Tokyo/Berkeley/SJTU — completed all four Erdős problems: 7 target theorems, 258 lemmas, 0 sorry, fully autonomous. The key insight: reliable research-level autoformalization needs agent durability (fault containment, statement fidelity), not just stronger provers. Lean 4 kernel verification throughout; code open on GitHub.

Google DeepMind's LEAP framework lifts general LLM formal theorem-proving from <10% to 70% on the new Lean-IMO-Bench, solves all 12 Putnam 2025 problems in Lean 4, and autonomously produces 5,000+ lines of kernel-verified code for a Knuth combinatorics subproblem. Lean compiler verification throughout; no NL seeding for Putnam results. Claim audit: closed-source competitors excluded; statement faithfulness not independently audited.

Goedel-Architect achieves 99.2% pass@1 on MiniF2F-test and 88.8% on PutnamBench using open-weight DeepSeek-V4-Flash — the strongest open-source formal theorem proving pipeline to date, at ~$294 for a full PutnamBench run (vs. ~$163,000 for Hilbert). Lean 4 kernel verification throughout, zero sorry. Claim audit: competition scores (IMO 4/6, Putnam 11/12) require NL-seeded mode, not fully autonomous.

SorryDB benchmarks AI provers on real sorry-closure tasks from 78 live GitHub Lean 4 projects. Competition-specialist Kimina Prover achieves only 1.0% pass@1; Goedel-Prover-V2 hits 2.7% — both trailing every general LLM. Gemini Flash agentic (with compiler error feedback) tops at 30.3%. The finding: competition-benchmark champions are nearly useless on real-world Lean formalization tasks.

DeepMind's AlphaProof Nexus agent resolved 9 open Erdős problems and 44 OEIS conjectures — all formally verified by the Lean 4 kernel with zero sorry placeholders. First large-scale open-problem resolution via AI + formal proof search. Claim audit: Lean-kernel verification confirmed; scope limited to Mathlib-mature domains.

An OpenAI reasoning model disproved the planar unit distance conjecture (Erdős 1946) on May 20, 2026. The proof uses Golod–Shafarevich theory from algebraic number theory and was verified by Gowers, Alon, Shankar, and Tsimerman. No Lean kernel formalization exists yet — LeanMarathon tried and explicitly failed.