Beyond the Library: An Agentic Framework for Autoformalizing Research Mathematics
Quick Answer
This paper introduces an agentic framework for autoformalizing research mathematics using general coding LLMs, outperforming smaller models in Lean 4.
Quick Take
This paper introduces an agentic framework for autoformalizing research mathematics using general coding LLMs, outperforming smaller models in Lean 4. The system dynamically extends type definitions and validates them before formalizing theorems, successfully producing machine-checked proofs for 32 PutnamBench problems and five ACM STOC papers.
Key Points
- Introduces an agentic framework for autoformalization using general coding LLMs.
- Successfully produced machine-checked proofs for 32 PutnamBench problems.
- Evaluated on five ACM STOC papers, formalizing main theorems and proofs.
- Dynamic type definition extension validated via a novel Auxiliary Lemma technique.
- All formalizations are publicly available at the provided GitHub link.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 31134v1 Announce Type: new Abstract: While Large Language Models (LLMs) have demonstrated exceptional capabilities in mathematical reasoning, they frequently produce subtle errors that evade human detection. Formal mathematical languages like Lean 4 offer mechanical proof checking, strongly motivating the need for autoformalization: the automatic translation of natural language mathematics into verifiable code.
Recent trends indicate that general-purpose LLMs, heavily optimized for standard programming, now outperform smaller models explicitly fine-tuned for Lean. Leveraging this shift, we introduce an agentic autoformalization framework powered by general coding LLMs. At the core of our system is an orchestrator that manages a pipeline tailored for research-level mathematics.
Because cutting-edge research frequently relies on concepts outside the scope of existing libraries like Mathlib, our system dynamically extends necessary type definitions and validates them via a novel Auxiliary Lemma technique before formalizing the primary theorems. We applied our approach to PutnamBench, producing machine-checked Lean proofs for a random sample of 32 problems.
Furthermore, we evaluate our system on five papers from the ACM Symposium on Theory of Computing (STOC) spanning combinatorics, communication complexity, mechanism design, and learning theory, successfully formalizing their main theorems and validating the generated formalizations with human experts; for all five we also formalize the proofs alongside the statements, and notably two of them are proved with no axioms beyond Lean's kernel. All of our formalizations are available at https://beyondthelibrary. github.
io/formal_arxiv.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Verification Horizon: No Silver Bullet for Coding Agent Rewards
As coding agents evolve, verifying solutions becomes more challenging than generating them, necessitating a focus on scalable, faithful, and robust verification methods. The study reveals that no fixed reward function can sustain effectiveness as model capabilities advance, emphasizing the need for verification to evolve alongside solution generation.