MIME: Multimodal Interactive Motion Encoder
Quick Answer
This paper shows that The Multimodal Interactive Motion Encoder (MIME) is a novel model for two-person interactive motion representation, achieving a 12.8% relative improvement in text-to-motion retrieval on the Inter-X benchmark.
Quick Take
MIME employs stream-based co-attention and curriculum contrastive training, enhancing semantic alignment in motion generation tasks across datasets.
Key Points
- MIME is the first dedicated multimodal encoder for two-person interactions.
- Achieved a 12.8% improvement in text-to-motion retrieval at 2,000 samples.
- Utilizes stream-based co-attention and explicit interaction features.
- Evaluated as a frozen auxiliary prior in TIMotion and InterMask.
- Improves semantic alignment while maintaining comparable FID metrics.
DeepSignal Analysis
What happened
The Multimodal Interactive Motion Encoder (MIME) is introduced as a model for two-person interactive motion representation. It achieves a 12.8% relative improvement in text-to-motion retrieval on the Inter-X benchmark, utilizing stream-based co-attention and curriculum contrastive training.
Key evidence
- MIME is designed specifically for two-person interactive motion, capturing both individual and shared dynamics.
- The model outperforms early and late fusion baselines on the Inter-X benchmark, achieving a 12.8% relative improvement in text-to-motion retrieval at a 2,000-sample gallery.
- MIME is evaluated as a frozen auxiliary prior within TIMotion and InterMask, improving semantic alignment metrics while maintaining comparable FID.
Why it matters
MIME's development reflects a growing need for advanced motion representation in applications such as animation, AR/VR, and embodied AI. By enhancing the alignment of language with actor dynamics and interactions, MIME could significantly improve the quality of motion generation tasks across various datasets.
What to watch
Paper Resources
📖 Reader Mode
~2 min readAbstract:Text-motion representation learning has advanced rapidly, with growing interest in multi person interactions for animation, AR/VR, and embodied AI. These settings require representations that align language with both individual actor dynamics and the relationships between actors. We introduce the Multimodal Interactive Motion Encoder (MIME), which, to our knowledge, represents the first dedicated multimodal encoder designed specifically for two person interactive motion. MIME captures individual and shared structure using stream based co-attention with explicit interaction features and curriculum based contrastive training. On Inter-X text-motion retrieval, MIME consistently outperforms early and late fusion baselines across gallery sizes, achieving a 12.8% relative improvement in text-to-motion R@1 at a 2,000-sample gallery. We further evaluate MIME as a frozen auxiliary prior within TIMotion and InterMask on the unseen InterHuman dataset. MIME improves semantic alignment metrics while maintaining comparable FID in TIMotion. These results show that interaction aware multimodal encoding improves multi person motion retrieval and transfers across datasets to support downstream motion generation.
| Comments: | Under review at WACV 2027 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.22702 [cs.CV] |
| (or arXiv:2607.22702v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.22702 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Addison Zucek [view email]
[v1]
Sat, 18 Jul 2026 20:08:31 UTC (6,946 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.