Efficacy of MRI-based deep learning algorithm for detecting acute ischemic stroke: evaluation among diverse readers.
Authors
Affiliations (6)
Affiliations (6)
- Department of Radiology, Eunpyeong St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea.
- Department of Radiology, Eunpyeong St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea. [email protected].
- Department of Radiology, Ilsan Paik Hospital, Inje University, Goyang, Korea.
- Siemens Healthineers AG, Erlangen, Germany.
- Siemens Medical Solutions USA, Inc., Princeton, NJ, USA.
- Siemens Healthineers Ltd., Seoul, Korea.
Abstract
The efficacy of an MRI-based deep learning algorithm (DLA) for detecting acute ischemic stroke (AIS) was evaluated across readers with diverse medical backgrounds, because DLA performance may be user-dependent. This retrospective, multi-reader, multi-case crossover study included 407 MRI scans obtained from a single institution between April and June 2021. Nine readers with different backgrounds- radiology residents (1-2 years of radiology training), clinicians (no radiology training), and board-certified non-neuroradiologists (completed residency training)-independently read MRI scans, both with and without DLA detection probability. The ground truth was established by consensus among three neuroradiologists. The area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, diagnostic confidence (0-4), and inter-reader agreement were compared between the groups with and without DLA. In total, 407 patients (mean age, 66 years ± 16; 200 men) with 95 AIS (23%) were evaluated. Clinicians had the lowest baseline performance scores. The DLA significantly improved clinicians' AUC (from 0.90 [95% CI: 0.82-0.99]; to 0.93 [0.87-0.99]; p < 0.01), sensitivity (from 0.77 [0.65-0.88]; to 0.88 [0.75-0.99]; p < 0.01), and diagnostic confidence (from 0.71 ± 1.42; to 0.83 ± 1.53; p < 0.01), and all readers' inter-reader agreement (p < 0.01). Specificity for clinicians (from 0.95 [0.86-0.99] to 0.93 [0.80-0.99]; p = 0.55) and the performance of residents and non-neuroradiologists were not significantly affected by DLA assistance. The DLA significantly improved the performance and diagnostic confidence of clinicians, the lowest-performing readers, and the inter-reader agreement of all readers in diagnosing AIS. Question What is the efficacy of an MRI-based deep learning algorithm in assisting various medical professionals in identifying acute ischemic stroke? Findings Among radiology residents, clinicians, and board-certified non-neuroradiologists, the algorithm significantly improved clinicians' performance and diagnostic confidence while also enhancing inter-reader agreement for all readers. Clinical relevance The deep learning algorithm significantly improves the detection performance and diagnostic confidence of clinicians, the lowest-performing readers, for acute ischemic stroke. Furthermore, the inter-reader agreement among various medical professionals has improved significantly.