Towards Verifiable Agentic Data Science: Solving Irregular TSQA Via Tool-Grounded Reasoning
Quick Answer
The IRTS-ToolBench introduces a benchmark of 1,700 questions across 10 task types to evaluate LLMs in irregular time series question answering (TSQA), addressing gaps in current benchmarks that assume regular sampling.
Quick Take
This tool aims to enhance the understanding of AI agents' performance under real-world conditions with asynchronous observations and informative missing values.
Key Points
- IRTS-ToolBench features 1,700 questions across 13 domains for evaluating irregular TSQA.
- Existing benchmarks fail to address irregular sampling in time series data.
- The benchmark supports independent use by researchers focusing on -based analysis.
- Standardized inputs and reproducible evaluation protocols are provided for consistency.
- Code for IRTS-ToolBench is available on GitHub.
Paper Resources
Source Excerpt
arXiv:2606. 15107v1 Announce Type: new Abstract: Time series data in real-world deployments is overwhelmingly irregular. Observations are asynchronous, missing values are informative rather than random, and sampling frequencies vary across sensors and operational windows. However, existing Time Series Question Answering (TSQA) benchmarks mostly assume regularly sampled inputs, leaving a fundamental gap in understanding how (LLMs) and AI agents perform under irregular conditions.
To bridge this gap, we introduce IRTS-ToolBench, a benchmark of 1,700 questions spanning 10 task types across 13 domains. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.