Accuracy of Diagnosis and Anticipation of Future Treatment in Pediatric Peripelvic Infections: The Role of Multisurgeon Review and Implications for Future Machine Learning Algorithms.
Authors
Affiliations (1)
Affiliations (1)
- From the Department of Orthopaedic Surgery, University of Utah, Salt Lake City, UT (Staten, Dr. Turtle, and Dr. Bellaire); the Department of Orthopaedic surgery, University of Michigan Health-West, Wyoming, MI (Dr. Warden); the Shriners Children's Twin Cities Hospital, Woodbury, MN (Dr. Lang); and the Department of Orthopaedics and Rehabilitation, University of Wisconsin, Madison WI (Noonan).
Abstract
MRI is commonly used to evaluate pelvic musculoskeletal infections. Limited "quick" MRI protocols enable timely imaging without intravenous contrast or sedation. This study examines the consistency of interpretation among pediatric orthopaedic surgeons when using quick versus full, contrast-enhanced MRI scans in cases of peripelvic musculoskeletal infection and explores these findings to inform future development and design of machine learning algorithms. Twenty-nine pediatric patients with full pelvis MRI with and without contrast and culture-positive peripelvic infection were retrospectively identified. Two deidentified files were created for each patient: one including all sequences and the other containing only the limited sequences included in our institution's quick MRI protocol. Three pediatric orthopaedic surgeons independently and sequentially evaluated the images, followed by group discussion to reach consensus on the primary diagnosis and management. Fleiss' Kappa (FK) statistic was calculated for each outcome. Moderate agreement in primary diagnosis was observed among reviewers using quick MRI sequences (Kappa = 0.488), and substantial agreement was seen with full sequences (0.684; P = 0.003). Inter-rater agreement on treatment recommendations was poor with both quick (0.09) and full (0.233) MRI (P = 0.046). No difference was found in team consensus diagnosis and final diagnosis between quick (0.523) and full (0.569) MRI (P = 0.662). Poor agreement was found between team treatment recommendations and actual treatment for both quick (0.182) and full (0.07) MRI (P = 0.254). Independent evaluation of limited, quick MRI sequences by pediatric orthopaedic surgeons showed more variability in diagnosis and treatment compared with full MRI review. When reviewed collaboratively, the diagnostic accuracy of quick MRI approached that of full MRI. Future artificial intelligence-based imaging interpretation platforms will benefit from multi-institutional collaboration to improve training data quality; use of ensemble learning techniques to reflect the diversity of multispecialist approaches; and incorporation of relevant clinical data to properly identify, triage, and direct treatment of complex pediatric musculoskeletal conditions.