MeddeX: A Retrieval-Augmented LLaMA-3 System with Gemini Vision for Evidence-Backed Medical Report Analysis
Quick Answer
This paper shows that MeddeX combines a fine-tuned LLaMA-3 model with Retrieval-Augmented Generation and Gemini Vision to enhance medical report analysis, achieving significant improvements in ROUGE and BLEU scores.
Quick Take
The system effectively processes various document formats and introduces features like report-scoped retrieval and automated longitudinal comparisons, marking a robust advancement in AI-assisted clinical decision support.
Key Points
- MeddeX utilizes a fine-tuned LLaMA-3 (8B) model for medical report analysis.
- Achieved ROUGE-1, ROUGE-2, and ROUGE-L improvements of 9.2%, 10.0%, and 11.8%, respectively.
- Introduces a novel garbage-text detection mechanism for corrupted documents.
- Supports text-native PDFs, scanned images, and radiological images through a hybrid extraction pipeline.
- Features a FastAPI backend and Supabase vector store for efficient data processing.
DeepSignal Analysis
What happened
MeddeX is a newly developed Medical Report Analyzer that integrates a fine-tuned LLaMA-3 model with Retrieval-Augmented Generation and Gemini Vision. It aims to improve the analysis of complex medical reports, achieving notable enhancements in evaluation metrics such as ROUGE and BLEU scores.
Key evidence
- The MeddeX system utilizes a QLoRA fine-tuned LLaMA-3 (8B) model, which is specifically designed for medical document analysis.
- Evaluation results show improvements in ROUGE-1 (+9.2%), ROUGE-2 (+10.0%), ROUGE-L (+11.8%), and BLEU (+71.4%) compared to the base LLaMA3-Med42-8B model.
- MeddeX can process various document formats, including text-native PDFs and scanned images, through an intelligent hybrid extraction pipeline.
Why it matters
The increasing complexity of medical reports poses challenges for healthcare professionals, leading to slow and error-prone manual reviews. MeddeX addresses these issues by providing accurate and concise summaries, which can enhance clinical decision-making and improve patient outcomes. The integration of advanced technologies like RAG and Gemini Vision represents a significant step forward in AI-assisted healthcare.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Modern healthcare generates increasingly complex medical reports that challenge clinical professionals in terms of time and interpretive burden. Manual review processes are slow, error-prone, and difficult to scale. This paper presents MeddeX, a Medical Report Analyzer that combines a QLoRA fine-tuned LLaMA-3 (8B) model with Retrieval-Augmented Gen eration (RAG) and a multimodal Gemini Vision layer to deliver accurate, concise, and evidence-backed summaries of medical documents. The system handles text-native PDFs, scanned image based PDFs, and direct radiological images through an intelligent hybrid extraction pipeline that includes a novel garbage-text detection mechanism for corrupted and UTF-16-encoded doc uments. Beyond single-report Q&A, MeddeX introduces report scoped retrieval for precise contextual grounding, an automated longitudinal report comparison engine, and a self-healing backfill pipeline that lazily processes legacy documents on demand. A FastAPI backend, Supabase vector store, and this http URL fron tend complete the end-to-end architecture. Evaluation against the base LLaMA3-Med42-8B model demonstrates consistent improvements across ROUGE-1 (+9.2%), ROUGE-2 (+10.0%), ROUGE-L (+11.8%), BLEU (+71.4%), and BERTScore (+0.8%), validating the benefit of domain-specific fine-tuning. MeddeX represents a practical and robust step toward AI-assisted clinical decision support.
| Comments: | 6 pages , 3 figures, published in ijsrem |
| Subjects: | Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| ACM classes: | I.2.7; I.2.10; H.3.3 |
| Cite as: | arXiv:2610.20632 [cs.CL] |
| (or arXiv:2610.20632v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20632 arXiv-issued DOI via DataCite |
|
| Journal reference: | IJSREM, International Journal of Scientific Research in Engineering and Management Volume 10 | 4| April 2026 |
Submission history
From: Rajdeep Pandhare [view email]
[v1]
Wed, 30 Sep 2026 13:27:05 UTC (41 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
MARS-Gov introduces a framework for detecting bureaucratic bias in Dutch government documents, achieving a new state-of-the-art F1 score of 0.880. This model outperforms existing zero-shot detectors by 20.2 points and reduces unnecessary interventions to just 2.5%. The framework's dynamic '10th juror' adapts to emerging biases, enhancing legal language processing.