TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning
Quick Answer
TAG-DLM introduces a masked diffusion language model that unifies textual reasoning and graph message passing for text-attributed graphs.
Quick Take
TAG-DLM introduces a masked diffusion language model that unifies textual reasoning and graph message passing for text-attributed graphs. It outperforms existing methods, including graph neural networks and LLM-based models, achieving up to 3.9 points improvement on TAG benchmarks across node classification and link prediction tasks without task-specific fine-tuning.
Key Points
- TAG-DLM integrates textual reasoning with graph message passing in a single model.
- Achieves state-of-the-art performance on TAG benchmarks across multiple tasks.
- Improves over the strongest baseline by up to 3.9 points.
- Supports node classification, link prediction, and cross-dataset transfer.
- Eliminates the need for target-specific fine-tuning.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 31166v1 Announce Type: new Abstract: Text-attributed graphs (TAGs), where each node carries a natural language description, require models to jointly reason over text and graph topology. Existing approaches often handle the two modalities separately: graph neural networks operate on shallow text features, while hybrids of LLMs and graphs use the language model mainly as a text encoder and delegate structure learning to a separate graph module.
We propose method that unifies textual reasoning and graph message passing within a masked diffusion language model, a language model with bidirectional attention and generative decoding. For each graph instance, method linearises a sampled local neighbourhood into a token sequence and injects graph structure through a topology attention mask, which realises message passing over the graph.
Because the diffusion language model can both interpret and generate text, the method adapts to different tasks simply by changing the prompt, supporting node classification, link prediction, and cross-dataset transfer with no target-specific fine-tuning. Experiments show that method outperforms graph neural networks, graph transformers, and LLM-based baselines on all three TAG benchmarks across two tasks, improving over the strongest baseline by up to 3. 9 points.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.