Behavioral Controllability of Agentic Models for Information Extraction: From Fixed Workflows to Reflective Agents
Quick Answer
This study evaluates the impact of agentic components like reflection and memory on large language model (LLM) workflows for information extraction.
Quick Take
By comparing fixed workflows with reflective agent variants, the research identifies an optimized agent condition (S2) that enhances task performance through dynamic tool selection and improved failure recovery mechanisms.
Key Points
- The study focuses on extracting datasets from scholarly PDFs using agents.
- An optimized agent condition (S2) utilizes richer PDF tools for enhanced performance.
- Process-level behaviors like tool execution and memory use were emphasized in evaluations.
- Agentic mechanisms were shown to alter system behavior and improve task completion.
- Failure modes observed inform the design of optimized agents for future tasks.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language model (LLM) agents are increasingly used for complex information-extraction tasks, yet it remains unclear whether agentic components such as reflection and memory lead to observable and controllable improvements over fixed LLM workflows. We study this question through conference-paper dataset extraction, where a system must identify datasets mentioned in scholarly PDFs and produce structured records. We compare a fixed workflow baseline with reflective agent variants and specify an optimized agent condition (S2) that extends the same task with richer PDF tools and dynamic tool selection. Our evaluation emphasizes process-level behavior--including tool execution, retries, reflection, memory use, runtime, and failure recovery--while treating extraction coverage and field completeness as secondary outcome measures. The paper characterizes when agentic mechanisms change system behavior, whether these changes improve task completion, and how the observed failure modes motivate an optimized agent design under the same evaluation harness.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.15715 [cs.AI] |
| (or arXiv:2607.15715v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.15715 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xingzhou Chen [view email]
[v1]
Fri, 17 Jul 2026 07:51:09 UTC (2,376 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.