Latest multimodal large language models show limitations on image-based radiology exam questions.
Key Details
- 1Researchers tested ChatGPT-4v and ChatGPT-4o on 222 image-based multiple-choice questions from national radiology board exams (2020 and 2024).
- 2These LLMs have been recently trained to process both text and images.
- 3Despite advancements, significant concerns remain regarding their reliability for diagnostic tasks in radiology.
- 4The potential of such models in radiology workflows, such as report generation and diagnostic support, is still under early investigation.
Why It Matters
As large language models gain capability for image analysis, assessing their reliability is crucial for safe deployment in radiology. Failures on board-style questions highlight the need for ongoing scrutiny before clinical trust is warranted.
Source
Radiology Business
Related News
•AuntMinnie
Study Evaluates LLMs for Automated PI-RADS Classification in Prostate MRI Reports
Large language models demonstrate promising performance in automating PI-RADS classification from structured prostate MRI reports, with some limitations in intermediate-risk lesions.
•AuntMinnie
Citizen Petition Proposes Streamlined FDA Pathway for Radiology AI
A petition urges the FDA to allow certain radiology AI software updates via postmarket study requirements instead of new 510(k) applications.
•AuntMinnie
Deep Learning Improves Dual-Energy CT Venography for Liver Transplants
Deep-learning image reconstruction significantly enhances image quality in dual-energy CT portal venography, aiding liver transplant planning.