Multimodal artificial intelligence (AI) has the potential to revolutionise healthcare by enabling the simultaneous processing and integration of various data types, including medical imaging, electronic health records, genomic information and real-time data. This review explores the current applications and future potential of multimodal AI across healthcare, with a particular focus on orthopaedic surgery. In presurgical planning, multimodal AI has demonstrated significant improvements in diagnostic accuracy and risk prediction, with studies reporting an Area under the receiving operator curve presenting good to excellent performance across various orthopaedic conditions. Intraoperative applications leverage advanced imaging and tracking technologies to enhance surgical precision, while postoperative care has been advanced through continuous patient monitoring and early detection of complications. Despite these advances, significant challenges remain in data integration, standardisation, and privacy protection. Technical solutions such as federated learning (allowing decentralisation of models) and edge computing (allowing data analysis to happen on site or closer to site instead of multipurpose datacenters) are being developed to address these concerns while maintaining compliance with regulatory frameworks. As this field continues to evolve, the integration of multimodal AI promises to advance personalised medicine, improve patient outcomes, and transform healthcare delivery through more comprehensive and nuanced analysis of patient data. Level of Evidence: Level V.

Cerebrovascular diseases (CVD) can lead to stroke and dementia. Stroke is the second leading cause of death world wide and dementia incidence is increasing by the year. There are several markers of CVD that are visible on brain imaging, including: white matter hyperintensities (WMH), acute and chronic ischaemic stroke lesions (ISL), lacunes, enlarged perivascular spaces (PVS), acute and chronic haemorrhagic lesions, and cerebral microbleeds (CMB). Brain atrophy also occurs in CVD. These markers are important for patient management and intervention, since they indicate elevated risk of future stroke and dementia. We systematically reviewed automated systems designed to support radiologists reporting on these CVD imaging findings. We considered commercially available software and research publications which identify at least two CVD markers. In total, we included 29 commercial products and 13 research publications. Two distinct types of commercial support system were available: those which identify acute stroke lesions (haemorrhagic and ischaemic) from computed tomography (CT) scans, mainly for the purpose of patient triage; and those which measure WMH and atrophy regionally and longitudinally. In research, WMH and ISL were the markers most frequently analysed together, from magnetic resonance imaging (MRI) scans; lacunes and PVS were each targeted only twice and CMB only once. For stroke, commercially available systems largely support the emergency setting, whilst research systems consider also follow-up and routine scans. The systems to quantify WMH and atrophy are focused on neurodegenerative disease support, where these CVD markers are also of significance. There are currently no openly validated systems, commercially, or in research, performing a comprehensive joint analysis of all CVD markers (WMH, ISL, lacunes, PVS, haemorrhagic lesions, CMB, and atrophy).

To evaluate the single and combined diagnostic performances of CT and MRI radiomics for diagnosis of acute pancreatitis (AP). We prospectively enrolled 78 patients (mean age 55.7 ± 17 years, 48.7 % male) diagnosed with AP between 2020 and 2022. Patients underwent contrast-enhanced CT (CECT) within 48-72 h of symptoms and MRI ≤ 24 h after CECT. The entire pancreas was manually segmented tridimensionally by two operators on portal venous phase (PVP) CECT images, T2-weighted imaging (WI) MR sequence and non-enhanced and PVP T1-WI MR sequences. A matched control group (n = 77) with normal pancreas was used. Dataset was randomly split into training and test, and various machine learning algorithms were compared. Receiver operating curve analysis was performed. The T2WI model exhibited significantly better diagnostic performance than CECT and non-enhanced and venous T1WI, with sensitivity, specificity and AUC of 73.3 % (95 % CI: 71.5-74.7), 80.1 % (78.2-83.2), and 0.834 (0.819-0.844) for T2WI (p = 0.001), 74.4 % (71.5-76.4), 58.7 % (56.3-61.1), and 0.654 (0.630-0.677) for non-enhanced T1WI, 62.1 % (60.1-64.2), 78.7 % (77.1-81), and 0.787 (0.771-0.810) for venous T1WI, and 66.4 % (64.8-50.9), 48.4 % (46-50.9), and 0.610 (0.586-0.626) for CECT, respectively.The combination of T2WI with CECT enhanced diagnostic performance compared to T2WI, achieving sensitivity, specificity and AUC of 81.4 % (80-80.3), 78.1 % (75.9-80.2), and 0.911 (0.902-0.920) (p = 0.001). The MRI radiomics outperformed the CT radiomics model to detect diagnosis of AP and the combination of MRI with CECT showed better performance than single models. The translation of radiomics into clinical practice may improve detection of AP, particularly MRI radiomics.

We aimed to develop and validate an Artificial Intelligence (AI) model that leverages CCTA and optical coherence tomography (OCT) images for automated analysis of plaque characteristics and coronary function. A total of 100 patients who underwent invasive coronary angiography, OCT, and CCTA before discharge were included in this study. The data were randomly divided into a training set (80 %) and a test set (20 %). The training set, comprising 21,471 tomography images, was used to train a deep-learning convolutional neural network. Subsequently, the AI model was integrated with flow reserve score calculation software developed by Ruixin Medical. The results from the test set demonstrated excellent agreement between the AI model and OCT analysis for calcified plaque (McNemar test, p = 0.683), non-calcified plaque (McNemar test, p = 0.752), mixed plaque (McNemar test, p = 1.000), and low-attenuation plaque (McNemar test, p = 1.000). Additionally, there was excellent agreement for deep learning-derived minimum lumen diameter (intraclass correlation coefficient [ICC] 0.91, p < 0.001), mean vessel diameter (ICC 0.88, p < 0.001), and percent diameter stenosis (ICC 0.82, p < 0.001). In diagnosing >50 % coronary stenosis, the diagnostic accuracy of the AI model surpassed that of conventional CCTA (AUC 0.98 vs. 0.76, p = 0.008). When compared with quantitative flow fraction, there was excellent agreement between QFR and AI-derived CT-FFR (ICC 0.745, p < 0.0001). Our AI model effectively provides automated analysis of plaque characteristics from CCTA images, with the analysis results showing strong agreement with OCT findings. Moreover, the CT-FFR automatically analyzed by the AI model exhibits high consistency with QFR derived from coronary angiography.

Gallbladder cancer (GBC) is the most common malignant tumor in the biliary system, characterized by high malignancy, aggressiveness, and poor prognosis. Early diagnosis holds paramount importance in ameliorating therapeutic outcomes. Presently, the clinical diagnosis of GBC primarily relies on clinical-radiological-pathological approach. However, there remains a potential for missed diagnosis and misdiagnose in the realm of clinical practice. We firstly analyzed the blood-based biomarkers, such as carcinoembryonic antigen and carbohydrate antigen 19-9. Subsequently, we evaluated the diagnostic performance of various imaging modalities, including ultrasound (US), endoscopic ultrasound (EUS), computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography/computed tomography (PET/CT) and pathological examination, emphasizing their strengths and limitations in detecting early-stage GBC. Furthermore, we explored the potential of emerging technologies, particularly artificial intelligence (AI) and liquid biopsy, to revolutionize GBC diagnosis. AI algorithms have demonstrated improved image analysis capabilities, while liquid biopsy offers the promise of non-invasive and real-time monitoring. However, the translation of these advancements into clinical practice necessitates further validation and standardization. The review highlighted the advantages and limitations of current diagnostic approaches and underscored the need for innovative strategies to enhance diagnostic accuracy of GBC. In addition, we emphasized the importance of multidisciplinary collaboration to improve early diagnosis of GBC and ultimately patient outcomes. This review endeavoured to impart fresh perspectives and insights into the early diagnosis of GBC.

To address the challenges of verifying MR-based attenuation correction (MRAC) in PET/MR due to CT positional mismatches and alignment issues, this study utilized a flatbed insert and arms-down positioning during PET/CT scans to achieve precise MR-CT matching for accurate MRAC evaluation. A validation dataset of 21 patients underwent whole-body [¹⁸F]FDG PET/CT followed by [¹⁸F]FDG PET/MR. A flatbed insert ensured consistent positioning, allowing direct comparison of four MRAC methods-four-tissue and five-tissue models with discrete and continuous μ-maps-against CT-based attenuation correction (CTAC). A deep learning-based framework, trained on a dataset of 300 patients, was used to generate synthesized-CTs from MR images, forming the basis for all MRAC methods. Quantitative analyses were conducted at the whole-body, region of interest, and lesion levels, with lesion-distance analysis evaluating the impact of bone proximity on standardized uptake value (SUV) quantification. Distinct differences were observed among MRAC methods in spine and femur regions. Joint histogram analysis showed MRAC-4 (continuous μ-map) closely aligned with CTAC. Lesion-distance analysis revealed MRAC-4 minimized bone-induced SUV interference (r = 0.01, p = 0.8643). However, tissues prone to bone segmentation interference, such as the spine and liver, exhibited greater SUV variability and lower reproducibility in MRAC-4 compared to MRAC-2 (2D bone segmentation, discrete μ-map) and MRAC-3 (3D bone segmentation, discrete μ-map). Using a flatbed insert, this study validated MRAC with high precision. Continuous μ-value MRAC method (MRAC-4) demonstrated superior accuracy and minimized bone-related SUV errors but faced challenges in reproducibility, particularly in bone-rich regions.

Large language models (LLMs) show promise in radiological diagnosis, but their performance may be affected by the context of the cases presented. Our purpose is to investigate how providing information about prior probabilities influences the diagnostic performance of an LLM in radiological quiz cases. We analyzed 322 consecutive cases from Radiology's "Diagnosis Please" quiz using Claude 3.5 Sonnet under three conditions: without context (Condition 1), informed as quiz cases (Condition 2), and presented as primary care cases (Condition 3). Diagnostic accuracy was compared using McNemar's test. The overall accuracy rate significantly improved in Condition 2 compared to Condition 1 (70.2% vs. 64.9%, p = 0.029). Conversely, the accuracy rate significantly decreased in Condition 3 compared to Condition 1 (59.9% vs. 64.9%, p = 0.027). Providing information that may influence prior probabilities significantly affects the diagnostic performance of the LLM in radiological cases. This suggests that LLMs may incorporate Bayesian-like principles and adjust the weighting of their diagnostic responses based on prior information, highlighting the potential for optimizing LLM's performance in clinical settings by providing relevant contextual information.

Alzheimer's disease (AD) is a progressive neurodegenerative disorder characterized by intracellular neurofibrillary tangles with tau protein and extracellular β-amyloid plaques. Early and accurate diagnosis is crucial for effective treatment and management. The purpose of this review is to investigate new technologies that improve diagnostic accuracy while looking at the current diagnostic criteria for AD, such as clinical evaluations, cognitive testing, and biomarker-based techniques. A thorough review of the literature was done in order to assess both conventional and contemporary diagnostic methods. Multimodal strategies integrating clinical, imaging, and biochemical evaluations were emphasised. The promise of current developments in biomarker discovery was also examined, including mass spectrometry and artificial intelligence. Current diagnostic approaches include cerebrospinal fluid (CSF) biomarkers, imaging tools (MRI, PET), cognitive tests, and new blood-based markers. Integrating these technologies into multimodal diagnostic procedures enhances diagnostic accuracy and distinguishes dementia from other conditions. New technologies that hold promise for improving biomarker identification and diagnostic reliability include mass spectrometry and artificial intelligence. Advancements in AD diagnostics underscore the need for accessible, minimally invasive, and cost-effective techniques to facilitate early detection and intervention. The integration of novel technologies with traditional methods may significantly enhance the accuracy and feasibility of AD diagnosis.

In clinical practice, distinguishing between spinal tuberculosis (STB) and spinal tumors (ST) poses a significant diagnostic challenge. The application of AI-driven large language models (LLMs) shows great potential for improving the accuracy of this differential diagnosis. To evaluate the performance of various machine learning models and ChatGPT-4 in distinguishing between STB and ST. A retrospective cohort study. A total of 143 STB cases and 153 ST cases admitted to Xiangya Hospital Central South University, from January 2016 to June 2023 were collected. This study incorporates basic patient information, standard laboratory results, serum tumor markers, and comprehensive imaging records, including Magnetic Resonance Imaging (MRI) and Computed Tomography (CT), for individuals diagnosed with STB and ST. Machine learning techniques and ChatGPT-4 were utilized to distinguish between STB and ST separately. Six distinct machine learning models, along with ChatGPT-4, were employed to evaluate their differential diagnostic effectiveness. Among the 6 machine learning models, the Gradient Boosting Machine (GBM) algorithm model demonstrated the highest differential diagnostic efficiency. In the training cohort, the GBM model achieved a sensitivity of 98.84% and a specificity of 100.00% in distinguishing STB from ST. In the testing cohort, its sensitivity was 98.25%, and specificity was 91.80%. ChatGPT-4 exhibited a sensitivity of 70.37% and a specificity of 90.65% for differential diagnosis. In single-question cases, ChatGPT-4's sensitivity and specificity were 71.67% and 92.55%, respectively, while in re-questioning cases, they were 44.44% and 76.92%. The GBM model demonstrates significant value in the differential diagnosis of STB and ST, whereas the diagnostic performance of ChatGPT-4 remains suboptimal.

To develop and validate a deep learning model using multimodal PET/CT imaging for detecting and classifying focal liver lesions (FLL). This study included 185 patients who underwent ¹⁸F-FDG PET/CT imaging at our institution from March 2022 to February 2023. We analyzed serological data and imaging. Liver lesions were segmented on PET and CT, serving as the "reference standard". Deep learning models were trained using PET and CT images to generate predicted segmentations and classify lesion nature. Model performance was evaluated by comparing the predicted segmentations with the reference segmentations, using metrics such as Dice, Precision, Recall, F1-score, ROC, and AUC, and compared it with physician diagnoses. This study finally included 150 patients, comprising 46 patients with benign liver nodules, 51 patients with malignant liver nodules, and 53 patients with no FLLs. Significant differences were observed among groups for age, AST, ALP, GGT, AFP, CA19-9and CEA. On the validation set, the Dice coefficient of the model was 0.740. For the normal group, the recall was 0.918, precision was 0.904, F1-score was 0.909, and AUC was 0.976. For the benign group, the recall was 0.869, precision was 0.862, F1-score was 0.863, and AUC was 0.928. For the malignant group, the recall was 0.858, precision was 0.914, F1-score was 0.883, and AUC was 0.979. The model's overall diagnostic performance was between that of junior and senior physician. This deep learning model demonstrated high sensitivity in detecting FLLs and effectively differentiated between benign and malignant lesions.