Stanford and Rad Partners developed a structured framework for pre-deployment evaluation of radiology AI models to guide purchasing decisions.
Key Details
- 1Framework was developed by Stanford University and Rad Partners, detailed in the American Journal of Roentgenology.
- 2A workgroup of four radiologists evaluated 13 AI models from one vendor (Aidoc) between 2022 and 2024.
- 3Nearly 89,000 exams across multiple sites were used in the assessment.
- 4Attributes for evaluating value included task tediousness, likelihood of radiologist oversight, and clinical impact of misses.
- 5Five tasks were rated as high value, five as medium, and two as low based on the framework.
Why It Matters
This framework provides radiology groups with a practical, evidence-based method to evaluate AI models before investment, addressing the gap between AI marketing claims and real-world outcomes. Adoption of such structured assessment tools can improve clinical effectiveness and ROI for AI integration in imaging practices.
Source
Radiology Business
Related News
•Radiology Business
Radiologists Prefer Domain-Specific AI for CT Report Generation
Radiologists show a clear preference for domain-specific AI models in generating accurate and timely CT report impressions.
•AuntMinnie
Radiology Receives Declining Share of Industry Research Funding
Radiologists received only 1.1% of industry-funded research payments in 2024, with a continuing downward trend.
•AuntMinnie
GPT-4o AI Matches Radiologists in Follow-Up Imaging Recommendations
GPT-4o matched the performance of experienced radiologists and surpassed residents in recommending follow-up imaging from routine radiology reports.