Knowledge-guided Disentanglement with Atomic Actions for Action Recognition
Quick Answer
This paper shows that The Knowledge-guided Disentanglement with Atomic Actions (KDA) model enhances action recognition by using Large Language Models to decompose actions into atomic components, achieving state-of-the-art performance on multi-label benchmarks.
Quick Take
The integration of a Knowledge Injection Module (KIM) and a Knowledge Disentanglement Module (KDM) significantly improves feature discriminability and can be adapted to existing methods.
Key Points
- KDA leverages fine-grained semantic knowledge for improved action representation.
- Introduces a Knowledge Injection Module (KIM) for integrating atomic action knowledge.
- Achieves state-of-the-art results on multi-label action recognition benchmarks.
- Knowledge Disentanglement Module (KDM) enhances semantic guidance for action disentanglement.
- KIM and KDM can be integrated into other action recognition methods.
DeepSignal Analysis
What happened
The Knowledge-guided Disentanglement with Atomic Actions (KDA) model aims to improve action recognition in complex scenes by utilizing Large Language Models (LLMs) to break down actions into atomic components. This approach enhances feature discriminability and achieves state-of-the-art performance on multi-label benchmarks. The model incorporates a Knowledge Injection Module (KIM) and a Knowledge Disentanglement Module (KDM) to refine action representations.
Key evidence
- The KDA model uses Large Language Models to decompose action labels into atomic actions, providing explicit spatial-temporal semantics.
- A Knowledge Injection Module (KIM) integrates atomic action knowledge into video features, enhancing the representation of actions.
- KDA demonstrates improved feature discriminability and achieves state-of-the-art performance on multi-label action recognition benchmarks.
Why it matters
This research addresses the limitations of existing action recognition methods that often rely on holistic representations, which struggle to capture fine-grained interactions in complex scenes. By introducing a structured approach that incorporates semantic knowledge, KDA could lead to more accurate action recognition systems. The ability to integrate KIM and KDM into existing methods may also enhance the adaptability of current technologies in the field.
Paper Resources
Source Excerpt
Action recognition in complex scenes often involves multiple concurrent fine-grained actions, making it challenging to model internal action structures. Most existing methods rely on holistic representations, which are insufficient for capturing subtle interactions and fine-grained semantics. While recent prompt-based approaches introduce disentanglement, they lack explicit semantic guidance, and methods based solely on visual or structured cues remain coarse-grained. In this paper, we propose K
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.