Risk-adjusted training and evaluation for breast cancer detection.
Authors
Affiliations (8)
Affiliations (8)
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; Medical Faculty Heidelberg, Heidelberg University, Im Neuenheimer Feld 672, 69120 Heidelberg, Germany. Electronic address: [email protected].
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; Faculty of Mathematics and Computer Science, Heidelberg University, Im Neuenheimer Feld 205, 69120 Heidelberg, Germany; German Cancer Research Center (DKFZ), Helmholtz Imaging, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; German Cancer Consortium (DKTK), Partner Site Heidelberg, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; Medical Faculty Heidelberg, Heidelberg University, Im Neuenheimer Feld 672, 69120 Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Im Neuenheimer Feld 280, 69120 Heidelberg, Germany.
- Institute of Radiology, Uniklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Ulmenweg 18, 91054 Erlangen, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; German Cancer Research Center (DKFZ), Helmholtz Imaging, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; Medical Faculty Heidelberg, Heidelberg University, Im Neuenheimer Feld 672, 69120 Heidelberg, Germany; Faculty of Mathematics and Computer Science, Heidelberg University, Im Neuenheimer Feld 205, 69120 Heidelberg, Germany; German Cancer Consortium (DKTK), Partner Site Heidelberg, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany; National Center for Tumor Diseases (NCT), Heidelberg University Hospital (UKHD) and German Cancer Research Center (DKFZ), Im Neuenheimer Feld 460, 69120 Heidelberg, Germany.
Abstract
Breast cancer detection, and broadly medical object detection, revolves around discovering and rating lesions. One of the most common ways of measuring performance is FROC (Free-response Receiver Operating Characteristic), which calculates sensitivity at predefined thresholds of false positives per case. However, depending on the clinical context, not all lesions might be of equivocal impact on the long-term outcome of a patient. Some lesions missed e.g. in screening might be detected in the subsequent screening round without impacting the clinical prognosis, whilst missing others might significantly detoriate prognosis and treatment pathways. It is therefore desirable to develop and include consideration of clinical prognosis/risk imbalance in the way machine learning models are developed and evaluated. In this work, we propose risk-adjusted FROC (raFROC), an adaptation of FROC that constitutes a first step on reflecting the underlying clinical need more accurately. Experiments on two independent breast magnetic resonance imaging (MRI) datasets with a total of 1535 lesions in 1735 subjects showcase the clinical potential of the proposed metric and its advantages over traditional evaluation methods. Additionally, by utilizing a risk-adjusted adaptation of focal loss (raFocal) we are able to improve the raFROC results and patient-level performance of nnDetection, at no expense of the regular FROC.