Agentic systems for breast cancer treatment recommendations
Quick Answer
This study evaluates agentic LLM systems for breast cancer treatment recommendations using 72 clinical cases.
Quick Take
This study evaluates agentic LLM systems for breast cancer treatment recommendations using 72 clinical cases. The best-performing model, Claude Opus 4.8 with the D&C+SA pipeline, achieved a score of 0.594 ± 0.025, but persistent errors indicate these systems are not yet suitable for unsupervised clinical use.
Key Points
- Evaluated seven pipelines, including single-LLM and architectures.
- and agent autonomy had mixed effects on performance.
- Persistent errors included incorrect recommendations and citation errors.
- Performance varied by clinical domain and disease stage.
- Findings suggest LLMs can aid in recommendations but need supervision.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language models (LLMs) are increasingly being explored for clinical decision support, but their reliability in complex oncology treatment planning remains unclear. We evaluated agentic LLM systems for breast cancer treatment recommendation generation using 72 real clinical cases across stages I to IV and 1,147 case-specific rubrics generated through Asymmetric Information Rubric Generation (AIRG), in which the rubric generator had access to real clinical decisions unavailable to the evaluated models. Seven pipelines were compared, including single-LLM baselines, tool-augmented systems, and multi-agent architectures with fact checking and autonomous subagent spawning. The best-performing configuration, Claude Opus 4.8 with the D&C+SA pipeline, achieved a global score of 0.594 $\pm$ 0.025. Tool use and increased agent autonomy had mixed effects, improving performance in some settings but degrading it in others. Performance varied by clinical domain and disease stage, and oncologist-led error analysis revealed persistent clinically relevant failures, including incorrect or missing recommendations, flawed justifications, citation errors, outdated claims, and overconfidence. These findings suggest that agentic LLM systems can generate clinically relevant breast cancer recommendations, but remain insufficient for unsupervised clinical use.
| Comments: | Under peer review. Source code available at: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.12051 [cs.CL] |
| (or arXiv:2607.12051v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.12051 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vinicius Anjos De Almeida [view email]
[v1]
Mon, 13 Jul 2026 18:09:50 UTC (1,084 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.