MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning
Quick Answer
MIRA-Math introduces a benchmark for mathematical reasoning where solvers must request a missing atomic fact to complete a problem.
Quick Take
MIRA-Math introduces a benchmark for mathematical reasoning where solvers must request a missing atomic fact to complete a problem. It contains 2,310 instances across 22 mathematical families, revealing that request success and final-answer accuracy are distinct challenges. The dataset and tools are provided for reproducible evaluation in AI research.
Key Points
- MIRA-Math features 2,310 instances from 22 mathematical families.
- Solvers must request one missing fact to derive the final answer.
- Request success and final-answer accuracy are independent metrics.
- The benchmark aids in evaluating minimal information requesting in AI.
- Tools for instance generation and validation are included for reproducibility.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and long-horizon dialogue. We introduce MIRA-Math, a benchmark for a narrower diagnostic capability: solving mathematical problems whose full latent state has a unique answer, but whose solver-facing view is missing exactly one necessary atomic fact. The solver must request the missing information in natural language under a strict budget and then integrate the returned fact into an exact final answer. A fixed constrained LLM responder sees only the dataset-provided atomic fact and must either offer the quoted fact when the request matches it, or decline otherwise. Thus, instance generation, typed hint specifications, validation, and final-answer verification are deterministic, while request metrics are measured under a fixed LLM-mediated responder channel. MIRA-Math contains 2{,}310 generated instances from 22 typed mathematical families spanning algebra, probability, linear systems, discrete structures, signal processing, Markov chains, circuits, interpolation, and numerical boundary-value problems. Experiments across frontier and small models show that request success and final-answer accuracy are separable: models may ask for the right fact yet fail the downstream computation, or fail before obtaining the canonical hint. We release generators, verifiers, prompts, run metadata, and dataset documentation to support reproducible evaluation of minimal information requesting in mathematical reasoning.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.07391 [cs.AI] |
| (or arXiv:2607.07391v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.07391 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Samer Saab Jr [view email]
[v1]
Wed, 8 Jul 2026 13:23:56 UTC (650 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Onnes: A Physics-Grounded LLM Simulator for Cryogenic Fault Diagnosis in Quantum Computing Infrastructure
Onnes is a physics-grounded digital twin simulator for dilution refrigerators, enhancing cryogenic fault diagnosis in quantum computing. It achieves a classification accuracy of 99.0% using few-shot demonstrations, matching a supervised ML classifier, while maintaining a low false alarm rate of 6.4% on real hardware.