General Covariant Action Modeling: Constructing Generalized Manifolds via Spatio-Temporal Decoupling
Quick Take
The Generalized Action Manifold (GAM) framework enhances embodied intelligence by enforcing general covariance through structural disentanglement, achieving robust generalization from limited data. By integrating GAM within a Vision-Language-Action architecture, it outperforms geometry-agnostic baselines, demonstrating superior transfer and robustness capabilities.
Key Points
- GAM enforces temporal and geometric invariance for robust action representation.
- Utilizes Arc-Length Parameterizer to decouple spatial and temporal dynamics.
- Maps trajectories to canonical 'world lines' for enhanced spatial generalizability.
- Empirical results show GAM outperforms traditional geometry-agnostic methods.
- Enables sparse demonstrations to densely populate a valid action manifold.
Article Content
From source RSS / original summaryarXiv:2606. 00110v1 Announce Type: new Abstract: Achieving robust generalization from limited data is a central challenge in embodied intelligence. Prevailing methods fail by regressing absolute coordinates, which violates the principle of general covariance. Fundamentally, this conflates the intrinsic task geometry with rigid execution patterns, binding policies to specific motion styles and fixed speeds.
To resolve this, we propose the Generalized Action Manifold (GAM) framework that enforces general covariance through structural disentanglement.
Specifically, GAM realizes the manifold by enforcing invariance across two orthogonal dimensions: (1) Temporal Invariance, utilizing an Arc-Length Parameterizer to orthogonalize the spatial path geometry from temporal dynamics, ensuring robustness to velocity variations; (2) Geometric Invariance, where a Schema-Affine-Factorization mechanism maps trajectories to canonical ``world lines'' in a pose-normalized coordinate frame.
This distinguishes invariant geometric schemas from affine modulations, ensuring spatial generalizability. By integrating GAM within a structured Vision-Language-Action (VLA) architecture, we enable sparse demonstrations to densely populate a continuous, valid action manifold. Empirical results demonstrate that GAM enables superior transfer and robustness capabilities, outperforming geometry-agnostic baselines.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
Evi-Steer introduces a novel evidential tuning framework for BiomedCLIP, enabling efficient fine-tuning with only 0.11% parameter updates. It significantly enhances performance in few-shot learning and domain shifts across 15 biomedical imaging datasets, demonstrating robustness for clinical applications.
