OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models
Quick Answer
OGPO introduces an Offline Goal-conditioned Policy Optimization framework for Vision-Language-Action models, enhancing post-training by focusing on key-state reachability rather than mere action imitation.
Quick Take
Experiments on LIBERO and MetaWorld demonstrate significant performance improvements, with OGPO showing greater robustness and goal-directed recoverability under distribution shifts.
Key Points
- OGPO relabels trajectories with semantic key states for better task progress.
- Shifts VLA post-training from action imitation to key-state learning.
- Consistently outperforms standard supervised fine-tuning in experiments.
- Demonstrates stronger robustness under distribution shifts and off-demonstration states.
- Accepted at NeurIPS 2026, indicating significant academic interest.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Vision-language-action (VLA) models have shown strong promise in robotic manipulation, yet their post-training still predominantly relies on trajectory-level supervised imitation over robot demonstrations. While effective for learning feasible actions, this paradigm supervises policies to reproduce expert actions at each step, biasing them toward demonstration-specific action paths and providing limited guidance for recovering toward task-progress states once execution deviates from demonstrated trajectories. To address this limitation, we propose OGPO, an Offline Goal-conditioned Policy Optimization framework for recoverable VLA post-training. Instead of treating offline demonstrations merely as step-wise action labels, OGPO adaptively relabels trajectories with semantic key states and constructs goal-conditioned process rewards to optimize the policy for reaching these task-progress states. In this way, OGPO shifts VLA post-training from trajectory-level action imitation to key-state reachability learning, enabling policies to acquire a more robust ability to complete goal states from feasible off-demonstration states without requiring additional online interaction. Experiments on LIBERO and MetaWorld show that OGPO consistently improves VLA policy performance over standard supervised fine-tuning. Moreover, evaluations on LIBERO-Plus and our proposed LIBERO-ReAct, a mid-execution perturbation benchmark, further demonstrate that OGPO achieves stronger robustness and goal-directed recoverability under distribution shifts and recoverable off-demonstration states.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.12914 [cs.AI] |
| (or arXiv:2610.12914v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12914 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xule Gao [view email]
[v1]
Fri, 9 Oct 2026 03:04:50 UTC (2,570 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →VEPA: Verified Evolving Formal Policy for Lifelong Agent Guardrails
VEPA introduces a framework for verified evolving formal policies for LLM agents, enhancing safety by adapting guardrails based on deployment feedback. It outperforms existing methods across benchmarks like ASSEBench-Safety and ASSEBench-Security, ensuring deterministic policy enforcement even under challenging conditions.