Automatic Extraction of Structured Information from Brain MRI Reports Using an Open-Weight Large Language Model
Quick Answer
This paper shows that The LLaMA 3.1 model demonstrates high performance in extracting structured information from Dutch brain MRI reports, achieving 90% accuracy for medial temporal atrophy and 93% for microbleed mentions.
Quick Take
Few-shot prompting significantly enhances numerical data extraction, indicating strong potential for large-scale neuroradiology research.
Key Points
- LLaMA 3.1 achieved 90% accuracy for medial temporal atrophy in MRI reports.
- Microbleed mentions were detected with 93% accuracy using the model.
- Few-shot prompting improved numerical variable extraction significantly.
- Performance metrics were evaluated across 947 Dutch neuroradiology reports.
- Challenges remain for location-specific variables despite high overall accuracy.
Paper Resources
Source Excerpt
arXiv:2606. 07721v1 Announce Type: new Abstract: Objectives: Automatic data extraction from free-text radiology reports enables large-scale research, but few studies assessed the performance of (LLMs) on Dutch neuroradiology reports. Methods: We analyzed 947 brain MRI reports from a tertiary memory clinic (2016-2021), authored by consultant neuroradiologists. Trained medical students annotated thirty variables; 100 reports were double-annotated to assess inter-rater reliability.
We evaluated the performance of the open-weight LLM LLaMA 3. 1 using different languages (Dutch vs. English translation) and few-shot prompting with different example selection strategies. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →AINTMA: Agentic AI Architecture for Autonomous Test Management with Generative Intelligence, Secure Cloud Communication and Adaptive Quality Analytics
AINTMA, an autonomous test management architecture utilizing six specialized AI agents, achieves 88.4% test prioritization accuracy and reduces defect escape rates from 8.3% to 2.1%. The system demonstrates a 340% ROI within nine months, showcasing the potential of agentic AI in enhancing software quality management in cloud environments.