Clinical Implementation of AI for Pulmonary Embolism Detection in over 30,000 CT Pulmonary Angiography Examinations.
Authors
Affiliations (5)
Affiliations (5)
- Northwell Health, 2000 Marcus Ave, Ste 300, New Hyde Park, NY 11042-1069.
- Department of Radiology, Donald and Barbara Zucker School of Medicine, Hofstra/Northwell, Hempstead, NY.
- Frank G. Zarb School of Business, Hofstra University, Hempstead, NY.
- Institute of Health System Science, Feinstein Institutes for Medical Research, Manhasset, NY.
- Harvey L. Nieman Health Policy Institute, Reston, Va.
Abstract
Purpose To quantify postimplementation concordance between a U.S. Food and Drug Administration-cleared artificial intelligence (AI) tool and AI-informed radiologists for pulmonary embolism (PE) detection on CT pulmonary angiography (CTPA), with real-time adjudication of discordances. Materials and Methods A PE AI tool (AIDOC, Tel Aviv, Israel) was retrospectively implemented in the clinic across an integrated network (August 9, 2021-February 20, 2023). Adult CTPAs underwent real-time AI analysis and radiologist interpretation. Radiologist-AI disagreements triggered adjudication by thoracic radiologists via the AI Quality Oversight Process; adjudicator diagnosis served as the reference standard for discordant cases. Concordance was measured and diagnostic performance of radiologists and AI was compared using adjudication for discordant cases. Results 32,501 CTPAs obtained from 29,492 patients (mean age, 62.4 years ± 18.6 [SD], 17,424 female) were evaluated. PE positivity was 9.93% (3,226/32,501). Overall concordance was 97.79% (95% confidence interval [CI]: 97.62-97.94%), higher for AI-negative than for AI-positive examinations (98.18% vs 93.75%; <i>P</i> < .001). Expert adjudication favored the radiologist in 88.73% of discordances. The rate of unique diagnosis by the interpreting radiologist (483/3,226, 14.97%) was approximately 19 times that of the AI tool alone (26/3,226, 0.81%). Concordance varied by PE features: acute versus chronic (87.34% vs 60.12%; <i>P</i> < .001) and location (central 95.79%, lobar/segmental 83.81%, subsegmental 58.62%; all <i>P</i> < .001). Conclusion In large-scale deployment, AI showed high concordance with radiologists and made meaningful contributions in discordant reviews = while expert oversight confirmed complementary roles and highlighted scenarios of radiologist-AI divergence. ©RSNA, 2026.