Artificial intelligence in emergency skeletal X-ray: post-deployment monitoring and clinical impact of incorrect AI results.
Authors
Affiliations (4)
Affiliations (4)
- Department of Life Sciences and Health, Oslo Metropolitan University, P.O.Box 4 St. Olavs Plass, 0130 Oslo, Norway.
- Department of Optometry, Radiography and Lighting Design, University of South-Eastern Norway, P.O. Box 4, 3199 Borre, Norway.
- Medical Diagnostics Clinic, Vestre Viken Health Trust, P.O. Box 800, 3004 Drammen, Norway.
- Medical Diagnostics Clinic, Vestre Viken Health Trust, P.O. Box 800, 3004 Drammen, Norway. Electronic address: [email protected].
Abstract
This study evaluates the impact of the artificial intelligence (AI) application BoneView<sup>TM</sup> within a radiographer-supervised clinical workflow and subsequent effects on patient management. We examine false AI findings to understand the clinical consequences in skeletal X-ray examinations of suspected fracture injuries. This retrospective study was conducted at Bærum Hospital, Norway, where BoneView<sup>TM</sup> was introduced within an AI-assisted radiographer-supervised emergency skeletal X-ray workflow in September 2023. The study included patients who were referred to emergency skeletal X-ray in January 2024. We calculated diagnostic accuracy of AI compared with AI-assisted radiologists, and we examined patient management following the introduction of the new AI-assisted workflow. The study included 1248 patients. AI alone demonstrated an overall sensitivity of 0.95 (95% CI 0.92 to 0.97) and specificity of 0.90 (95% CI 0.88 to 0.92) when compared to AI-assisted radiologists. Compared to AI's initial suggestion, radiographer supervision altered the subsequent patient pathway in 20% of the cases, with the likelihood overriding AI being higher when AI results were later found to be incorrect (i.e. false positives or false negatives). Notably, the AI-assisted workflow only led to premature discharge of one patient who required treatment, and this patient was recalled to hospital for treatment following the radiologist's report. AI demonstrates strong diagnostic accuracy with high sensitivity and specificity in emergency skeletal X-ray examinations. Radiographers may play a crucial role in mitigating erroneous clinical decisions due to false AI-results and ensure appropriate patient management.