Does Theory of Mind Improvement Really Benefit Human-AI Interactions? Empirical Findings from Interactive Evaluations
Quick Answer
This study reveals that improving Theory of Mind (ToM) in Large Language Models (LLMs) does not guarantee enhanced performance in human-AI interactions.
Quick Take
This study reveals that improving Theory of Mind (ToM) in Large Language Models (LLMs) does not guarantee enhanced performance in human-AI interactions. The authors propose a new interactive evaluation paradigm and find that traditional benchmarks fail to capture the complexities of dynamic interactions, emphasizing the need for interaction-based assessments to develop socially aware LLMs.
Key Points
- Traditional ToM benchmarks focus on static assessments, neglecting dynamic human-AI interactions.
- The study introduces an interactive ToM evaluation paradigm with perspective and metric shifts.
- Four ToM enhancement techniques were systematically evaluated using real-world datasets.
- Static benchmark improvements do not always lead to better dynamic interaction performance.
- The findings stress the importance of interaction-based assessments for future LLM development.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Improving the Theory of Mind (ToM) capability of Large Language Models (LLMs) is crucial for effective social interactions between these AI models and humans. However, the existing benchmarks often measure ToM capability improvement through story-reading, multiple-choice questions from a third-person perspective, while ignoring the first-person, dynamic, and open-ended nature of human-AI (HAI) interactions. To directly examine how ToM improvement techniques benefit HAI interactions, we first proposed the new paradigm of interactive ToM evaluation with both perspective and metric shifts. Next, following the paradigm, we conducted a systematic study of four representative ToM enhancement techniques using both four real-world datasets and a user study, covering both goal-oriented tasks (e.g., coding, math) and experience-oriented tasks (e.g., counseling). Our findings reveal that improvements on static benchmarks do not always translate to better performance in dynamic HAI interactions. This paper offers critical insights into ToM evaluation, showing the necessity of interaction-based assessments in developing next-generation, socially aware LLMs for HAI symbiosis.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.15205 [cs.AI] |
| (or arXiv:2605.15205v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.15205 arXiv-issued DOI via DataCite |
Submission history
From: Nanxu Gong [view email]
[v1]
Tue, 28 Apr 2026 15:38:31 UTC (7,139 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Procedural Memory Distillation: Online Reflection for Self-Improving Language Models
Procedural Memory Distillation (PMD) enhances reinforcement learning by converting cross-episode signals into reusable memory, improving Qwen3-8B and OLMo3-Instruct-7B models by 3.8-5.5% on SCIKNOWEVAL and 7.9-13.6% on . The co-evolution of policy and memory allows for more effective self-supervision, demonstrating significant performance gains when both components are active.