A deep learning system for non-invasive breast cancer diagnosis with multimodal data.
Authors
Affiliations (22)
Affiliations (22)
- School of Biomedical Engineering and State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai, China.
- Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, China.
- Department of Radiology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Kunming, China.
- Department of Ultrasound, Wenzhou People's Hospital, Wenzhou, China.
- Department of Radiology, The Affiliated Hospital of Southwest Medical University, Luzhou, China.
- Department of Radiology, The First Affiliated Hospital of Kunming Medical University, Kunming, China.
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China.
- Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China.
- Department of Radiology, Guizhou Provincial People's Hospital, Guiyang, China.
- Hangzhou Medical College, Hangzhou, China.
- Department of Radiology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
- Department of Radiology, Second Xiangya Hospital of Central South University, Changsha, China.
- Department of Ultrasound, The Affiliated Hospital of Southwest Medical University, Luzhou, China.
- Department of Ultrasound Medicine, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Kunming, China.
- Department of Breast Surgery, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Kunming, China.
- Department of Radiology, Guizhou Provincial People's Hospital, Guiyang, China. [email protected].
- Department of Radiology, Affiliated Hangzhou First People's Hospital, Westlake University School of Medicine, Hangzhou, China. [email protected].
- School of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China. [email protected].
- Department of Radiology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Kunming, China. [email protected].
- School of Biomedical Engineering and State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai, China. [email protected].
- Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China. [email protected].
- Shanghai Clinical Research and Trial Center, Shanghai, China. [email protected].
Abstract
Early and accurate diagnosis of breast cancer is critical for minimizing needle biopsies and enhancing patient outcomes and requires effective integration of multimodal information. In this article, we introduce a breast cancer intelligent non-invasive diagnosis system (BINDS) to integrate multimodal medical imaging data for breast cancer risk assessment and subtype classification. BINDS uses a two-stage diagnostic approach to match the clinical workflow, where an initial assessment with ultrasound and/or mammography is performed, followed by a more comprehensive multimodal diagnosis incorporating magnetic resonance imaging. In addition, a new radiology-pathology alignment mechanism is proposed to facilitate extraction of pathology-relevant features from radiological images. BINDS is developed and validated with a diverse dataset of 27,048 participants from 8 centres and 7 public datasets. Importantly, BINDS supports flexible combinations of input modalities during training and validation. Notably, BINDS attains an area under the receiver operating characteristic curve of 0.973, and can assist radiologists in reducing biopsies of benign lesions by up to 32.4%. These findings highlight the potential of BINDS to advance breast cancer diagnosis by enabling precise and adaptable decision-making across diverse clinical scenarios and resource settings.