RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning
Quick Answer
This paper shows that RADIANT-PET integrates a voxel-level segmentation model with a large language model for enhanced PET/CT lesion classification, significantly reducing false positives.
Quick Take
RADIANT-PET integrates a voxel-level segmentation model with a large language model for enhanced PET/CT lesion classification, significantly reducing false positives. The framework outperforms traditional methods, especially when radiology reports are included, demonstrating improved lesion detection and clinical alignment.
Key Points
- RADIANT-PET uses a permissive segmentation stage to identify candidate lesions.
- A large language model classifies lesions as true positives or false positives.
- Reinforcement learning optimizes lesion classification and anatomical site assignment.
- The framework shows consistent performance improvements over image-only baselines.
- Results indicate enhanced clinical interpretation and reduced false positives.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Accurate lesion segmentation in PET/CT is critical for oncology, yet remains challenging because physiologic tracer uptake and artifacts can mimic malignant signal. We present RADIANT-PET, a reasoning-augmented framework that couples a high-sensitivity voxel-level segmentation model with lesion-level large language model (LLM) adjudication. Candidate uptake regions are generated with a deliberately permissive segmentation stage, then converted into structured textual descriptions that summarize uptake intensity, morphology, and regional and global anatomical context. An LLM classifies each candidate as true lesion vs. false positive, optionally leveraging the radiology report as additional clinical context. To strengthen lesion-level reasoning, we further optimize a local LLM via reinforcement learning using Group Relative Policy Optimization, rewarding correct lesion classification and anatomically concordant site assignment. Across AutoPET and an OSU test cohort, RADIANT-PET consistently outperforms strong image-only baselines, with the largest improvements observed when radiology reports are provided. Overall, these results demonstrate that LLM-based lesion-level reasoning adds a novel reasoning layer beyond conventional segmentation, suppressing physiologic false positives and aligning voxel-level predictions with clinical interpretation. The project repository is available at: this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.28392 [cs.CV] |
| (or arXiv:2606.28392v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.28392 arXiv-issued DOI via DataCite |
Submission history
From: Simeng Zhu [view email]
[v1]
Tue, 23 Jun 2026 18:05:55 UTC (408 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.