New multimodal large language models (LLMs) like OpenAI o3 and Gemini 2.5 Pro demonstrated significant advancements in answering Japanese radiology board exam questions, particularly with image input.
Key Details
- 1Eight LLMs were tested on the Japan Diagnostic Radiology Board Examination (JDRBE).
- 2OpenAI o3 achieved 67% accuracy (text-only) and 72% with image input.
- 3Gemini 2.5 Pro also showed notable accuracy improvements with image data.
- 4Both OpenAI o3 and Gemini 2.5 Pro received higher legitimacy scores from radiologist raters than some competitors.
- 5The test set included 233 questions and 477 images (184 CT, 159 MRI, 15 x-ray, 90 nuclear medicine).
- 6Image input statistically improved diagnostic accuracy for several models.
Why It Matters
This study marks the first demonstration of statistically significant improvement in LLM diagnostic accuracy with image input on a radiology board exam, signaling meaningful progress for AI-assisted radiological training and assessment.
Source
AuntMinnie
Related News
•AuntMinnie
AI Model Uses Ultrasound to Assess Fetal Lung Maturity
Researchers demonstrated an AI model's strong accuracy in measuring fetal lung maturity from ultrasound images.
•AuntMinnie
AI Model Predicts Dosimetry for Lu-177 PSMA Therapy Using PET/CT
A machine learning PET/CT model shows promise for predicting radiation dose prior to Lu-177 PSMA therapy in prostate cancer patients.
•AuntMinnie
AI Advances in Ultrasound Highlighted at AIUM 2026 Keynote
AI is increasingly enhancing ultrasound imaging, clinical workflows, and education, though challenges in trust and implementation remain.