Concurrent AI-human interaction in prostate cancer MRI interpretation: More hype than help?
Authors
Affiliations (6)
Affiliations (6)
- Department of Advanced Biomedical Sciences, University of Naples Federico II, Naples, Italy.
- Department of Radiology, Santa Maria Delle Grazie Hospital, ASL Napoli 2 Nord, Pozzuoli, Italy.
- Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy.
- Department of Medicine, Surgery and Dentistry, University of Salerno, Baronissi, Italy. [email protected].
- Institute of Radiology, Department of Medicine (DMED), University of Udine, University Hospital S. Maria della Misericordia, Azienda Sanitaria Universitaria Friuli Centrale (ASUFC), Udine, Italy.
- Paul Strickland Scanner Centre, Mount Vernon Cancer Centre, Northwood, UK.
Abstract
We evaluated a commercial artificial intelligence (AI) system as a concurrent decision-support tool for clinically significant prostate cancer (csPCa) detection. In our retrospective study, consecutive patients underwent multiparametric MRI for clinical suspicion of PCa. All scans were reviewed by six readers with varying expertise (two expert radiologists, > 1,000 cases; two basic radiologists, 400‒1,000 cases; and two residents), with and without AI assistance. Intra-/inter-reader agreements and the impact of AI-assistance on patient-level csPCa scores and diagnostic performance, as well as benefit-to-harm ratios, were assessed. The population consisted of 100 patients with a 26% prevalence of csPCa. There was no improvement in inter-reader agreement with AI-assistance versus without (Fleiss κ 0.573 and 0.584, respectively). Residents were most likely to change PI-RADS scores on AI-assisted readings compared to basic and expert radiologists (19, 9, and 7 changes, respectively). Overall, there was no significant difference in area under the receiving operating characteristic curve between AI-assisted and AI-unassisted readings (0.87 versus 0.86; p = 0.734). At a PI-RADS ≥ 3 threshold, sensitivity was slightly lower with AI (0.87 versus 0.89), while specificity (0.73), positive predictive value (0.53-0.54), and negative predictive value (0.94-0.95) remained similar. Subgroup analyses showed no significant differences in diagnostic performance. A slight increase in grade selectivity and selective biopsy avoidance rate was observed among experts and residents, respectively, with AI-assisted readings when applying a PI-RADS cutoff of 3 or PSA density ≥ 0.15 ng/mL/mL. AI did not significantly improve diagnostic accuracy across readers of varying expertise, with minor impacts on benefit-to-harm ratios. We found that AI support in prostate MRI did not significantly improve diagnostic accuracy across readers of varying experience, highlighting the need for further research to optimize AI integration and define its most clinically meaningful roles in prostate cancer detection. Residents were most prone to PI-RADS score modifications after AI-assisted readings compared to AI-unassisted and expert readers. There was no significant difference in diagnostic performance metrics between AI-assisted and unassisted readings. A slight improvement in grade selectivity among experts and in selective biopsy avoidance among residents was observed during AI-assisted readings for biopsy recommendations.