MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion
Quick Answer
MGDT introduces a novel MLLM-Guided Diffusion Transformer for Multimodal Knowledge Graph Completion, enhancing entity inference by utilizing a Relation-Adaptive Mixture-of-Experts for semantic routing.
Quick Take
Experiments demonstrate that MGDT significantly outperforms existing methods across three benchmark datasets, addressing issues of noisy and inconsistent multimodal feature integration.
Key Points
- MGDT employs Relation-Adaptive Semantic Routing Mixture-of-Experts for effective cue selection.
- Utilizes a frozen Multimodal as a semantic anchor for alignment.
- Achieves superior performance on three benchmark datasets compared to strong baselines.
- Addresses challenges of noisy multimodal features in existing MKGC methods.
- Implements an align-then-diffuse paradigm for improved entity generation.
DeepSignal Analysis
What happened
MGDT is a new framework designed for Multimodal Knowledge Graph Completion (MKGC). It employs a Relation-Adaptive Mixture-of-Experts for selecting relevant multimodal features and uses a frozen Multimodal Large Language Model for semantic alignment. Experiments indicate that MGDT outperforms existing methods on three benchmark datasets.
Key evidence
- MGDT utilizes a Relation-Adaptive Semantic Routing Mixture-of-Experts module to enhance multimodal semantic transformation paths.
- The framework employs a frozen Multimodal Large Language Model as a semantic anchor to align multimodal representations.
- Experiments conducted on three benchmark datasets show that MGDT consistently outperforms strong baseline methods.
Why it matters
The introduction of MGDT addresses significant challenges in MKGC, particularly the issues of noisy and inconsistent multimodal feature integration. By improving the selection and alignment of multimodal cues, MGDT aims to enhance the accuracy of entity inference, which is crucial for applications in AI and knowledge representation.
What to watch
Paper Resources
Source Excerpt
Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues. Existing diffusion-based MKGC methods usually denoise directly on raw multimodal features. Such a design forces the denoiser to simultaneously perform relation-dependent cue selection, cross-modal semantic alignment, and structure-aware entity generation, which introduces noisy and semantically inconsistent conditions for diffusion and consequently leads to suboptimal compl
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Automatic Ordinary Differential Equations Discovery For Biological Systems Using Powered Agentic System
The MEDA system utilizes large language models and symbolic regression to autonomously discover ordinary differential equations for biological systems, achieving strong structural recovery and biologically plausible models. It outperforms existing methods by integrating domain knowledge and mechanistic constraints, demonstrating effective retrieval and extrapolation capabilities.