Detecting Performance Drift in AI Models for Medical Image Analysis Using CUSUM Chart.
Authors
Affiliations (7)
Affiliations (7)
- Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, 20993, MD, USA.
- Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, 20993, MD, USA. [email protected].
- Rockefeller Neuroscience Institute, West Virginia University, 33 Medical Center Dr, Morgantown, 26506, WV, USA.
- Vanderbilt Lung Institute, Vanderbilt University Medical Center, 1211 Medical Center Dr, Nashville, 37232, TN, USA.
- Department of Computer Science, University of Miami, 1320 S Dixie Hwy, Coral Gables, 33146, FL, USA.
- Department of Radiology, University of Miami, 1320 S Dixie Hwy, Coral Gables, 33146, FL, USA.
- Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, 20993, MD, USA. [email protected].
Abstract
The use of artificial intelligence (AI) models to assist clinical decisions for diagnosis and prediction has been shown to improve patient outcomes and clinical decision-making. However, the performance of an AI model deployed in a clinical setting can vary over time or between initial evaluations and clinical use because of data drift. Monitoring the performance of a deployed AI model may help to (i) ensure the AI model performs as expected in the clinical environment and (ii) detect performance deviations and alert stakeholders to these deviations. In this study, we investigate how a change in the performance of an AI model caused by an abrupt data drift can be detected using a cumulative sum (CUSUM) control chart. We demonstrate the use of CUSUM for computer-aided breast cancer detection using data from the publicly available Emory Breast Imaging Dataset (EMBED). Our results indicate that, when the magnitude of the drift is 1.5 times the standard deviation of the performance metric, CUSUM is able to detect changes in the test negativity rate of an AI model within an average of 5 days following the onset of performance drift, with a long duration ( <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>∼</mo></math> 293 days) between false alarms. Our results also indicate that when the average number of samples is 40/day, leading to a reasonable standard deviation for the performance metric, almost no false alarms are expected in a period of 60 days, with the choice of CUSUM parameters that lead to a small true alarm detection delay. We also demonstrate that CUSUM can be applied for monitoring the performance of AI models even when ground-truth labels are not available. We evaluate the robustness of the CUSUM chart to non-normal distributions and show that when tuned to detect relatively small parameter shifts, the CUSUM chart can be robust to non-normal distributions. We demonstrate that the sensitivity of CUSUM can be controlled to balance the mean time between false alarms and the delay in detecting a true change in performance. We conclude that CUSUM, which is a well-studied statistical process control method, can easily be adapted for monitoring the performance of AI models targeted for assisting in medical diagnosis.