AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions
Quick Answer
AutoRPA leverages LLMs to automate GUI tasks by converting ReAct-style interactions into efficient RPA functions, achieving an 82-96% reduction in token usage and enhancing runtime efficiency.
Quick Take
This framework combines a translator-builder pipeline and a hybrid repair strategy, demonstrating significant improvements in task execution across various GUI environments.
Key Points
- AutoRPA reduces token usage by 82% to 96% compared to traditional reasoning.
- The framework automates GUI tasks by synthesizing robust RPA functions from ReAct interactions.
- It features a translator-builder pipeline for converting hard-coded actions into soft-coded procedures.
- A hybrid repair strategy enhances code verification by combining RPA execution with ReAct fallback.
- Experiments show significant improvements in runtime efficiency and reusability across multiple environments.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large Language Model (LLM) based agents have demonstrated proficiency in multi-step interactions with graphical user interfaces (GUIs). While most research focuses on improving single-task performance, practical scenarios often involve repetitive GUI tasks for which invoking LLM reasoning repeatedly, i.e., the ReAct paradigm, is inefficient. Prior to LLMs, traditional Robotic Process Automation (RPA) offers runtime efficiency but demands significant manual effort to develop and maintain. To bridge this gap, we propose AutoRPA, a framework that automatically distills the decision logic of ReAct-style agents into robust RPA functions. AutoRPA introduces two core innovations: (1) A translator-builder pipeline, where a translator agent converts hard-coded ReAct actions into soft-coded procedures, and a builder agent synthesizes robust RPA functions via retrieval-augmented generation over multiple trajectories; (2) A hybrid repair strategy during code verification, combining RPA execution with ReAct-based fallback for iterative refinement. Experiments across multiple GUI environments demonstrate that RPA functions generated by AutoRPA successfully solve similar tasks while reducing token usage by 82% to 96%, significantly improving runtime efficiency and reusability.
| Comments: | Accepted in ICML 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.21082 [cs.AI] |
| (or arXiv:2605.21082v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.21082 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Minghao Chen [view email]
[v1]
Wed, 20 May 2026 12:17:43 UTC (10,069 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.