Impact of spectrum bias on deep learning-based stroke MRI analysis.
Authors
Affiliations (9)
Affiliations (9)
- University Hospital Copenhagen - Herlev and Gentofte, Department of Radiology, Denmark; Department of Clinical Medicine, University of Copenhagen, Denmark; Radiology AI Testcenter (RAIT.dk), Denmark. Electronic address: [email protected].
- University Hospital Copenhagen - Herlev and Gentofte, Department of Radiology, Denmark; Radiology AI Testcenter (RAIT.dk), Denmark.
- University Hospital Copenhagen - Herlev and Gentofte, Department of Radiology, Denmark.
- University Hospital Copenhagen - Herlev and Gentofte, Department of Radiology, Denmark; Novo Nordisk A/S, Søborg, Denmark.
- Department of Clinical Medicine, University of Copenhagen, Denmark; University Hospital Copenhagen - Herlev and Gentofte, Department of Neurology, Denmark.
- University Hospital Copenhagen - Herlev and Gentofte, Department of Radiology, Denmark; Department of Clinical Medicine, University of Copenhagen, Denmark; Radiology AI Testcenter (RAIT.dk), Denmark.
- University of Copenhagen, Department of Computer Science, Denmark.
- Department of Clinical Medicine, University of Copenhagen, Denmark; University Hospital Copenhagen - Rigshospitalet, Department of Brain and Spinal Cord Injury, Denmark; University Hospital Copenhagen - Herlev and Gentofte, Department of Neurology, Denmark.
- Department of Clinical Medicine, University of Copenhagen, Denmark; Radiology AI Testcenter (RAIT.dk), Denmark; University Hospital Copenhagen - Bispebjerg and Frederiksberg, Department of Radiology, Denmark.
Abstract
To evaluate spectrum bias in stroke MRI analysis by excluding cases with uncertain acute ischemic lesions (AIL) and examining patient, imaging, and lesion factors associated with these cases. This single-center retrospective observational study included adults with brain MRIs for suspected stroke between January 2020 and April 2022. Diagnostic uncertain AIL were identified through reader disagreement or low certainty grading by a radiology resident, a neuroradiologist, and the original radiology report consisting of various neuroradiologists. A commercially available deep learning tool analyzing brain MRIs for AIL was evaluated to assess the impact of excluding uncertain cases on diagnostic odds ratios. Patient-related, MRI acquisition-related, and lesion-related factors were analyzed using the Wilcoxon rank sum test, χ2 test, and multiple logistic regression. The study was approved by the National Committee on Health Research Ethics. In 989 patients (median age 73 (IQR: 59-80), 53% female), certain AIL were found in 374 (38%), uncertain AIL in 63 (6%), and no AIL in 552 (56%). Excluding uncertain cases led to a four-fold increase in the diagnostic odds ratio (from 68 to 278), while a simulated case-control design resulted in a six-fold increase compared to the full disease spectrum (from 68 to 431). Independent factors associated with uncertain AIL were MRI artifacts, smaller lesion size, older lesion age, and infratentorial location. Excluding uncertain cases leads to a four-fold overestimation of the diagnostic odds ratio. MRI artifacts, smaller lesion size, infratentorial location, and older lesion age are associated with uncertain AIL and should be accounted for in validation studies.