PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents
Quick Answer
PolyWorkBench introduces a benchmark for evaluating multilingual long-horizon LLM agents across 67 tasks in five domains.
Quick Take
Empirical results reveal significant performance degradation in state-of-the-art when handling multilingual workflows, emphasizing the need for better modeling of language variation and procedural decision-making.
Key Points
- PolyWorkBench evaluates LLM agents on multilingual tasks in commerce, legal analysis, and more.
- The benchmark includes 67 tasks requiring iterative reasoning and external tool invocation.
- State-of-the-art LLMs show significant performance drops in multilingual settings compared to monolingual.
- A hybrid evaluation framework combines structural grading and LLM-based semantic assessment.
- Multilinguality complicates reasoning and execution, necessitating joint modeling in .
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external environments. However, most existing benchmarks implicitly assume a monolingual setting, where the entire execution process, including reasoning, tool invocation, and output generation, is conducted within a single language. In contrast, real-world applications often involve multilingual inputs and outputs within a unified workflow, yet the interaction between multilinguality and agentic execution remains underexplored. In this work, we introduce PolyWorkBench, a benchmark for evaluating LLM agents on multilingual long-horizon workplace workflows. PolyWorkBench consists of 67 tasks across five domains, including commerce, knowledge work, legal analysis, localization, and manufacturing, where agents must process heterogeneous multilingual inputs, perform iterative reasoning, invoke external tools, and produce structured outputs. To enable comprehensive evaluation, we propose a hybrid framework that combines structural grading, executable verification, and LLM-based semantic assessment. This design allows us to capture both functional correctness and linguistic consistency across complex workflows. Empirical results show that state-of-the-art LLM agents suffer significant performance degradation in multilingual workflow settings compared to monolingual counterparts. Our analysis suggests that multilinguality introduces compounding effects across reasoning and execution steps, highlighting the importance of jointly modeling language variation and procedural decision-making in agent evaluation.
| Comments: | 15 Pages, 6 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.06008 [cs.AI] |
| (or arXiv:2607.06008v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.06008 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hongliang Li [view email]
[v1]
Tue, 7 Jul 2026 08:50:09 UTC (2,256 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.