MADS: Multi-Agent Dialog Simulation for Robust Conversational Agents
Quick Answer
This paper shows that MADS (Multi-Agent Dialog Simulation) introduces a synthetic data generation pipeline to enhance task-oriented conversational agents by simulating adversarial dialogues.
Quick Take
It constructs MADS-Bench, a benchmark with 1,095 conversations across six domains, where 66% involve failure-prone scenarios, and employs a two-stage training approach combining supervised fine-tuning and reinforcement learning to improve model robustness.
Key Points
- MADS generates structured, diverse dialogue trajectories to train robust conversational agents.
- MADS-Bench includes 1,095 conversations, with 66% featuring failure-prone scenarios.
- The two-stage training combines supervised fine-tuning and reinforcement learning for better performance.
- Existing benchmarks underrepresent adversarial conversations, limiting agent robustness.
- MADS aims to improve agent performance under non-cooperative user behavior.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Task-oriented conversational agents remain fragile under realistic conditions, especially when users exhibit non-cooperative behavior. Existing function-calling benchmarks underrepresent adversarial conversation trajectories, thereby limiting the training resources available for developing robust agents. We present MADS (Multi-Agent Dialog Simulation), a synthetic data generation pipeline that produces structured, diverse dialogue trajectories. We inject adversarial conversation, producing validated conversation trajectories that capture realistic, unhappy paths. Using MADS, we construct MADS-Bench: a benchmark spanning six domains, 55 personas, and 1095 conversations, with approximately 66% of conversations containing failure-prone conversation scenarios. We evaluate the robustness of the trained models using a two-stage training approach that combines supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR).
| Comments: | Submitted to Interspeech in February 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.12757 [cs.CL] |
| (or arXiv:2610.12757v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12757 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Arkajyoti Chakraborty [view email]
[v1]
Thu, 8 Oct 2026 22:08:10 UTC (420 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
MARS-Gov introduces a framework for detecting bureaucratic bias in Dutch government documents, achieving a new state-of-the-art F1 score of 0.880. This model outperforms existing zero-shot detectors by 20.2 points and reduces unnecessary interventions to just 2.5%. The framework's dynamic '10th juror' adapts to emerging biases, enhancing legal language processing.