Voice-controlled super-resolution ultrasound imaging and reporting powered by multimodal large language models.
Authors
Affiliations (8)
Affiliations (8)
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
- Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Key Laboratory of Diabetes Mellitus, Shanghai Clinical Center for Diabetes, Shanghai, China.
- Department of Critical Care Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
- Department of Radiology, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
- Department of Neurosurgery, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Jiao Tong University, Shanghai, China. [email protected].
- Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Key Laboratory of Diabetes Mellitus, Shanghai Clinical Center for Diabetes, Shanghai, China. [email protected].
- State Key Laboratory of Ocean Engineering, School of Ocean and Civil Engineering; Department of Radiology, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China. [email protected].
- Faculty of the SDU-SSPU Joint Program in Biomedical Engineering, Sanda University and Shanghai Polytechnic University, Shanghai, China. [email protected].
Abstract
Super-resolution ultrasound imaging (SRUI) surpasses the diffraction limit of conventional ultrasound, enabling visualization of microvascular architecture and hemodynamics with potential applications in neurology, oncology, and cardiology. However, clinical adoption remains limited by complex parameter optimization, subjective interpretation, and time-consuming workflows. We present a multimodal artificial intelligence framework that integrates a custom SRUI platform with large language models of DeepSeek-R1 for natural language processing and of MiniCPM-V for image recognition. Clinicians issue voice commands to initiate imaging tasks, which are translated into acquisition parameters, including temporal windows and adaptive microbubble filtration. The system performs super-resolution reconstruction, extracts quantitative vascular metrics, and generates structured diagnostic reports incorporating relevant clinical context. Filtration thresholds were dynamically determined using the Microbubble Similarity Score. Structured reports were generated within approximately four minutes. Evaluation by fourteen clinicians demonstrated good structural integrity and standardized terminology. This framework streamlines SRUI workflows and supports AI-assisted, clinically contextualized super-resolution ultrasound imaging. Trial registration: Chinese Clinical Trial Registry ChiCTR2100048361 registered July 6, 2021.