FAA-Net: Fetal abdominal anomaly diagnosis in prenatal ultrasound via LLM-enhanced multi-instance learning.
Authors
Affiliations (15)
Affiliations (15)
- Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen, China.
- Shenzhen Luohu People's Hospital (The Third Affiliated Hospital of Shenzhen University), Shenzhen, China.
- School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, China.
- Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong, China.
- The First Affiliated Hospital of Bengbu Medical University, Bengbu, China.
- Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China.
- Northwest Women's and Children's Hospital, Xian, China.
- First Hospital of China Medical University, Shenyang, China.
- Women's Hospital of Nanjing Medical University, Nanjing, China.
- Maternal and Child Health Hospital of Hubei Province, Wuhan, China.
- Qilu Hospital of Shandong University, Jinan, China.
- Center for Medical Ultrasound, The Affiliated Suzhou Hospital of Nanjing Medical University, Suzhou Municipal Hospital, Nanjing Medical University, Suzhou, China.
- Northwest Women's and Children's Hospital, Xian, China. Electronic address: [email protected].
- School of Artificial Intelligence, Shenzhen University, Shenzhen, China. Electronic address: [email protected].
- Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; School of Artificial Intelligence, Shenzhen University, Shenzhen, China; School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, China. Electronic address: [email protected].
Abstract
As a critical prenatal screening tool, ultrasound (US) heavily depends on operator experience, leading to both challenging and rare fetal malformation diagnoses as well as diagnostic variability. Thus, developing an automated fetal anomaly detection system can reduce operator variability and expedite clinical analysis. Currently, most existing methods perform independent image-based predictions on different planes and use simple pooling strategies for case-level prediction, disregarding inter-image correlations. Although multi-instance learning (MIL) can model inter-image correlations, it suffers from attention dispersion as the number of images per case increases, limiting the model's ability to capture spatial and morphological differences across anomaly categories. To address these limitations, we propose FAA-Net for diagnosing fetal abdominal anomalies. (1) We introduce a Mixture of View Experts module that captures spatial features from specialized sub-expert networks. (2) We develop an LLM-Enhanced Feature Selection module that mines the rich medical knowledge embedded in LLMs to select diagnostically relevant images, enabling the model to better focus and capture category-specific patterns. (3) A simple attention mechanism is used to capture inter-image spatial and morphological correlations. Extensively validated on a multi-center prenatal fetal abdominal ultrasound dataset containing 2,732 cases (77,833 images across 5 categories) and the public TMED-2 dataset, our proposed FAA-Net outperforms state-of-the-art competitors. Code is available at: Github.