MIME: Multimodal Interactive Motion Encoder
Quick Answer
This paper shows that The Multimodal Interactive Motion Encoder (MIME) is a novel model for two-person interactive motion representation, achieving a 12.8% relative improvement in text-to-motion retrieval on the Inter-X benchmark.
Quick Take
MIME employs stream-based co-attention and curriculum contrastive training, enhancing semantic alignment in motion generation tasks across datasets.
Key Points
- MIME is the first dedicated multimodal encoder for two-person interactions.
- Achieved a 12.8% improvement in text-to-motion retrieval at 2,000 samples.
- Utilizes stream-based co-attention and explicit interaction features.
- Evaluated as a frozen auxiliary prior in TIMotion and InterMask.
- Improves semantic alignment while maintaining comparable FID metrics.
DeepSignal Analysis
What happened
The Multimodal Interactive Motion Encoder (MIME) is introduced as a model for two-person interactive motion representation. It achieves a 12.8% relative improvement in text-to-motion retrieval on the Inter-X benchmark, utilizing stream-based co-attention and curriculum contrastive training.
Key evidence
- MIME is designed specifically for two-person interactive motion, capturing both individual and shared dynamics.
- The model outperforms early and late fusion baselines on the Inter-X benchmark, achieving a 12.8% relative improvement in text-to-motion retrieval at a 2,000-sample gallery.
- MIME is evaluated as a frozen auxiliary prior within TIMotion and InterMask, improving semantic alignment metrics while maintaining comparable FID.
Why it matters
MIME's development reflects a growing need for advanced motion representation in applications such as animation, AR/VR, and embodied AI. By enhancing the alignment of language with actor dynamics and interactions, MIME could significantly improve the quality of motion generation tasks across various datasets.
What to watch
Paper Resources
Source Excerpt
Text-motion representation learning has advanced rapidly, with growing interest in multi person interactions for animation, AR/VR, and . These settings require representations that align language with both individual actor dynamics and the relationships between actors. We introduce the Multimodal Interactive Motion Encoder (MIME), which, to our knowledge, represents the first dedicated multimodal encoder designed specifically for two person interactive motion. MIME captures individual
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.