Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics
Quick Answer
The study evaluates a ReAct-style agentic setup combining LLMs with SageMath, achieving an average performance gain of 9.7 percentage points on the RealMath benchmark.
Quick Take
Qwen 3.7-Max shows the highest benefit from SageMath, while GPT-5.5 achieves a 75.2% solve rate, indicating a promising direction for automated conjecture discovery in computational mathematics.
Key Points
- SageMath integration leads to substantial performance gains across evaluated models.
- Qwen 3.7-Max benefits the most from SageMath access.
- GPT-5.5 achieves the highest solve rate of 75.2% among tool-enabled configurations.
- A multi-step post-processing and validation pipeline enhances problem set quality.
- The project aims to assist mathematicians in computational exploration.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Recent advances in AI for Mathematics have focused largely on autoformalization and theorem proving, leaving the role of Computer Algebra Systems (CAS) in agentic LLM workflows underexplored. We propose a ReAct-style agentic setup that combines LLM reasoning with verifiable feedback from SageMath, together with Context7 for the up-to-date documentation. We evaluate this agentic setup across frontier models for solving research-level mathematical problems from the RealMath benchmark in a setting that emulates a computational-mathematics research loop. We also propose a refinement to the RealMath benchmark by introducing a multi-step post-processing procedure and a multi-stage validation pipeline, both of which improve the quality and reliability of the extracted problem set. Our experiments reveal substantial performance gains from SageMath access across all evaluated models on +9.7~pp on average, the gains range from 1.5~pp to 27.8~pp and narrow the gap between open-weight and closed models. Qwen~3.7-Max benefits from SageMath the most, while GPT-5.5 achieves the highest solve rate of $75.2\%$ and the lowest token usage among tool-enabled configurations. Our findings suggest that CAS-augmented agents represent a promising direction for assisting mathematicians in computational exploration, and we believe that this work is a step towards automated conjecture discovery. The project repository is available online.
| Comments: | 37 pages, 16 figures, accepted to 3rd AI for Math Workshop at ICML 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.06820 [cs.AI] |
| (or arXiv:2607.06820v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.06820 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pavel Snopov [view email]
[v1]
Tue, 7 Jul 2026 21:29:59 UTC (6,560 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.