Multimodal medical image fusion (MMIF) aims to integrate images from different modalities to produce a comprehensive image that enhances medical diagnosis by accurately depicting organ structures, tissue textures, and metabolic information. Capturing both the unique and complementary information across multiple modalities simultaneously is a key research challenge in MMIF. To address this challenge, this paper proposes a novel image fusion method, MMIF-AMIN, which features a new architecture that can effectively extract these unique and complementary features. Specifically, an Invertible Dense Network (IDN) is employed for lossless feature extraction from individual modalities. To extract complementary information between modalities, a Multi-scale Complementary Feature Extraction Module (MCFEM) is designed, which incorporates a hybrid attention mechanism, convolutional layers of varying sizes, and Transformers. An adaptive loss function is introduced to guide model learning, addressing the limitations of traditional manually-designed loss functions and enhancing the depth of data mining. Extensive experiments demonstrate that MMIF-AMIN outperforms nine state-of-the-art MMIF methods, delivering superior results in both quantitative and qualitative analyses. Ablation experiments confirm the effectiveness of each component of the proposed method. Additionally, extending MMIF-AMIN to other image fusion tasks also achieves promising performance.

Postoperative papillary thyroid cancer (PTC) patients often have enlarged cervical lymph nodes due to inflammation or hyperplasia, which complicates the assessment of recurrence or metastasis. This study aimed to explore the diagnostic capabilities of computed tomography (CT) imaging and radiomic analysis to distinguish the recurrence of cervical lymph nodes in patients with PTC postoperatively. A retrospective analysis of 194 PTC patients who underwent total thyroidectomy was conducted, with 98 cases of cervical lymph node recurrence and 96 cases without recurrence. Using 3D Slicer software, Regions of Interest (ROI) were delineated on enhanced venous phase CT images, analyzing 302 positive and 391 negative lymph nodes. These nodes were randomly divided into training and validation sets in a 3:2 ratio. Python was used to extract radiomic features from the ROIs and to develop radiomic models. Univariate and multivariate analyses identified statistically significant risk factors for cervical lymph node recurrence from clinical data, which, when combined with radiomic scores, formed a nomogram to predict recurrence risk. The diagnostic efficacy and clinical utility of the models were assessed using ROC curves, calibration curves, and Decision Curve Analysis (DCA). This study analyzed 693 lymph nodes (302 positive and 391 negative) and identified 35 significant radiomic features through dimensionality reduction and selection. The three machine learning models, including the Lasso regression, Support Vector Machine (SVM), and RF radiomics models, showed.

This study aimed to explore the diagnostic performance of ultrasound S-Detect in differentiating Breast Imaging-Reporting and Data System (BI-RADS) 4 breast nodules ≤ 20 mm and > 20 mm. Between November 2020 and November 2022, a total of 382 breast nodules in 312 patients were classified as BI-RADS-4 by conventional ultrasound. Using pathology results as the gold standard, we applied receiver operator characteristics (ROC), sensitivity (SE), specificity (SP), accuracy (ACC), positive predictive value (PPV), and negative predictive value (NPV) to analyze the diagnostic value of BI-RADS, S-Detect, and the two techniques in combination (Co-Detect) in the diagnosis of BI-RADS 4 breast nodules ≤ 20 mm and > 20 mm. There were 382 BI-RADS-4 nodules, of which 151 were pathologically confirmed as malignant, and 231 as benign. In lesions ≤ 20 mm, the SE, SP, ACC, PPV, NPV, and area under the curve (AUC) of the BI-RADS group were 77.27%, 89.73%, 85.71%, 78.16%, 89.24%, 0.835, respectively. SE, SP, ACC, PPV, NPV, and AUC of the S-Detect group were 92.05%, 78.92%, 83.15%, 67.50%, 95.43%, 0.855, respectively. SE, SP, ACC, PPV, NPV, and AUC of the Co-Detect group were 89.77%, 93.51%, 92.31%, 86.81%, 95.05%, 0.916, respectively. The differences of SE, ACC, NPV, and AUC between the BI-RADS group and the Co-Detect group were statistically significant (P < 0.05). In lesions > 20 mm, SE, SP, ACC, PPV, NPV, and AUC of the BI-RADS group were 88.99%, 89.13%, 88.99%, 91.80%, 85.42%, 0.890, respectively. SE, SP, ACC, PPV, NPV, and AUC of the S-Detect group were 98.41%, 69.57%, 86.24%, 81.58%, 96.97%, 0.840, respectively. SE, SP, ACC, PPV, NPV, and AUC of the Co-Detect group were 98.41%, 91.30%, 95.41%, 93.94%, 97.67%, 0.949, respectively. A total of 166 BI-RADS 4 A nodules were downgraded to category 3 by Co-Detect, with 160 (96.4%) confirmed as benign and 6 (all ≤ 20 mm) as false negatives. Conversely, 25 nodules were upgraded to 4B, of which 19 (76.0%) were malignant. The difference in AUC between the BI-RADS group and the Co-Detect group was statistically significant (P < 0.05). S-Detect combined with BI-RADS is effective in the differential diagnosis of BI-RADS 4 breast nodules ≤ 20 mm and > 20 mm. However, its performance is particularly pronounced in lesions ≤ 20 mm, where it contributes to a significant reduction in unnecessary biopsies.

Survival analysis utilizing multiple longitudinal medical images plays a pivotal role in the early detection and prognosis of diseases by providing insight beyond single-image evaluations. However, current methodologies often inadequately utilize censored data, overlook correlations among longitudinal images measured over multiple time points, and lack interpretability. We introduce SurLonFormer, a novel Transformer-based neural network that integrates longitudinal medical imaging with structured data for survival prediction. Our architecture comprises three key components: a Vision Encoder for extracting spatial features, a Sequence Encoder for aggregating temporal information, and a Survival Encoder based on the Cox proportional hazards model. This framework effectively incorporates censored data, addresses scalability issues, and enhances interpretability through occlusion sensitivity analysis and dynamic survival prediction. Extensive simulations and a real-world application in Alzheimer's disease analysis demonstrate that SurLonFormer achieves superior predictive performance and successfully identifies disease-related imaging biomarkers.

Generative artificial intelligence (AI) has been playing an important role in various domains. Leveraging its high capability to generate high-fidelity and diverse synthetic data, generative AI is widely applied in diagnostic tasks, such as lung cancer diagnosis using computed tomography (CT). However, existing generative models for lung cancer diagnosis suffer from low efficiency and anatomical imprecision, which limit their clinical applicability. To address these drawbacks, we propose Lung-DDPM+, an improved version of our previous model, Lung-DDPM. This novel approach is a denoising diffusion probabilistic model (DDPM) guided by nodule semantic layouts and accelerated by a pulmonary DPM-solver, enabling the method to focus on lesion areas while achieving a better trade-off between sampling efficiency and quality. Evaluation results on the public LIDC-IDRI dataset suggest that the proposed method achieves 8$\times$ fewer FLOPs (floating point operations per second), 6.8$\times$ lower GPU memory consumption, and 14$\times$ faster sampling compared to Lung-DDPM. Moreover, it maintains comparable sample quality to both Lung-DDPM and other state-of-the-art (SOTA) generative models in two downstream segmentation tasks. We also conducted a Visual Turing Test by an experienced radiologist, showing the advanced quality and fidelity of synthetic samples generated by the proposed method. These experimental results demonstrate that Lung-DDPM+ can effectively generate high-quality thoracic CT images with lung nodules, highlighting its potential for broader applications, such as general tumor synthesis and lesion generation in medical imaging. The code and pretrained models are available at https://github.com/Manem-Lab/Lung-DDPM-PLUS.

Computed tomography angiography (CTA) imaging is essential to evaluate and analyse complex abdominal and thoraco-abdominal aortic aneurysms. However, CTA analyses are labour intensive, time consuming, and prone to interphysician variability. Fully automatic volume segmentation (FAVS) using artificial intelligence with deep learning has been validated for infrarenal aorta imaging but requires further testing for thoracic and visceral aorta segmentation. This study assessed FAVS accuracy against physician controlled manual segmentation (PCMS) in the descending thoracic aorta, visceral abdominal aorta, and visceral vasculature. This was a retrospective, multicentre, observational cohort study. Fifty pre-operative CTAs of patients with abdominal aortic aneurysm were randomly selected. Comparisons between FAVS and PCMS and assessment of inter- and intra-observer reliability of PCMS were performed. Volumetric segmentation performance was evaluated using sensitivity, specificity, Dice similarity coefficient (DSC), and Jaccard index (JI). Visceral vessel identification was compared by analysing branchpoint coordinates. Bland-Altman limits of agreement (BA-LoA) were calculated for proximal visceral diameters (excluding duplicate renals). FAVS demonstrated performance comparable with PCMS for volumetric segmentation, with a median DSC of 0.93 (interquartile range [IQR] 0.03), JI of 0.87 (IQR 0.05), sensitivity of 0.99 (IQR 0.01), and specificity of 1.00 (IQR 0.00). These metrics are similar to interphysician comparisons: median DSC 0.93 (IQR 0.07), JI 0.87 (IQR 0.12), sensitivity 0.90 (IQR 0.08), and specificity 1.00 (IQR 0.00). FAVS correctly identified 99.5% (183/184) of visceral vessels. Branchpoint coordinates for FAVS and PCMS were within the limits of CTA spatial resolution (Δx -0.33 [IQR 2.82], Δy 0.61 [IQR 4.85], Δz 2.10 [IQR 4.69] mm). BA-LoA for proximal visceral diameter measurements showed reasonable agreement: FAVS vs. PCMS mean difference -0.11 ± 5.23 mm compared with interphysician variability of 0.03 ± 5.27 mm. FAVS provides accurate, efficient segmentation of the thoracic and visceral aorta, delivering performance comparable with manual segmentation by expert physicians. This technology may enhance clinical workflows for monitoring and planning treatments for complex abdominal and thoraco-abdominal aortic aneurysms.

Passive acoustic mapping (PAM) is a promising tool for monitoring acoustic cavitation activities in the applications of ultrasound therapy. Data-adaptive beamformers for PAM have better image quality compared with time exposure acoustics (TEA) algorithms. However, the computational cost of data-adaptive beamformers is considerably expensive. In this work, we develop a deep beamformer based on a generative adversarial network that can switch between different transducer arrays and reconstruct high-quality PAM images directly from radiofrequency ultrasound signals with low computational cost. The deep beamformer was trained on a dataset consisting of simulated and experimental cavitation signals of single and multiple microbubble clouds measured by different (linear and phased) arrays covering 1-15 MHz. We compared the performance of the deep beamformer to TEA and three different data-adaptive beamformers using simulated and experimental test dataset. Compared with TEA, the deep beamformer reduced the energy spread area by 27.3%-77.8% and improved the image signal-to-noise ratio by 13.9-25.1 dB on average for the different arrays in our data. Compared with the data-adaptive beamformers, the deep beamformer reduced the computational cost by three orders of magnitude achieving 10.5 ms image reconstruction speed in our data, while the image quality was as good as that of the data-adaptive beamformers. These results demonstrate the potential of the deep beamformer for high-resolution monitoring of microbubble cavitation activities for ultrasound therapy.

ObjectivePoor outcomes in acute respiratory distress syndrome (ARDS) can be alleviated with tools that support early diagnosis. Current machine learning methods for detecting ARDS do not take full advantage of the multimodality of ARDS pathophysiology. We developed a multimodal deep learning model that uses imaging data, continuously collected ventilation data, and tabular data derived from a patients electronic health record (EHR) to make ARDS predictions. Materials and MethodsA chest radiograph (x-ray), at least two hours of ventilator waveform (VWD) data within the first 24 hours of intubation, and EHR-derived tabular data were used from 220 patients admitted to the ICU to train a deep learning model. The model uses pretrained encoders for the x-rays and ventilation data and trains a feature extractor on tabular data. Encoded features for a patient are combined to make a single ARDS prediction. Ablation studies for each modality assessed their effect on the models predictive capability. ResultsThe trimodal model achieved an area under the receiver operator curve (AUROC) of 0.86 with a 95% confidence interval of 0.01. This was a statistically significant improvement (p<0.05) over single modality models and bimodal models trained on VWD+tabular and VWD+x-ray data. Discussion and ConclusionOur results demonstrate the potential utility of using deep learning to address complex conditions with heterogeneous data. More work is needed to determine the additive effect of modalities on ARDS detection. Our framework can serve as a blueprint for building performant multimodal deep learning models for conditions with small, heterogeneous datasets.

BackgroundKawasaki disease (KD) is an acute, pediatric vasculitis associated with coronary artery abnormality (CAA) development. Echocardiography at month 1 post-diagnosis remains the standard for CAA surveillance despite limitations, including patient distress and increased healthcare burden. With declining CAA incidence due to improved treatment, the need for routine follow-up imaging is being reconsidered. This study aimed to develop and externally validate models for predicting CAA development and guide the need for echocardiography. MethodsThis study used two prospective multicenter Japanese registries: PEACOCK for model development and internal validation, and Post-RAISE for external validation. The primary outcome was CAA at the month 1 follow-up, defined as a maximum coronary artery Z score (Zmax) [&ge;] 2. Twenty-nine clinical, laboratory, echocardiographic, and treatment-related variables obtained within one week of diagnosis were selected as predictors. The models included simple models using the previous Zmax as a single predictor, logistic regression models, and machine learning models (LightGBM and XGBoost). Their discrimination, calibration, and clinical utility were assessed. ResultsAfter excluding patients without outcome data, 4,973 and 2,438 patients from PEACOCK and Post-RAISE, respectively, were included. The CAA incidence at month 1 was 5.5% and 6.8% for the respective group. For external validation, a simple model using the Zmax at week 1 produced an area under the curve of 0.79, which failed to improve by more than 0.02 after other variables were added or more complex models were used. Even the best-performing models with a highly sensitive threshold failed to reduce the need for echocardiography at month 1 by more than 30% while maintaining the number of undiagnosed CAA cases to less than ten. The predictive performance declined considerably when the Zmax was omitted from the multivariable models. ConclusionsThe Zmax at week 1 was the strongest predictor of CAA at month 1 post-diagnosis. Even advanced models incorporating additional variables failed to achieve a clinically acceptable trade-off between reducing the need for echocardiography and reducing the number of undiagnosed CAA cases. Until superior predictors are identified, echocardiography at month 1 should remain the standard practice. Clinical PerspectiveO_ST_ABSWhat Is New?C_ST_ABSO_LIThe maximum Z score on echocardiography one week after diagnosis was the strongest of 29 variables for predicting coronary artery abnormalities (CAA) in patients with Kawasaki disease. C_LIO_LIEven the most sensitive models had a suboptimal ability to predict CAA development and reduce the need for imaging studies, suggesting they have limited utility in clinical decision-making. C_LI What Are the Clinical Implications?O_LIUntil more accurate predictors are found or imaging strategies are optimized, performing echocardiography at one-month follow-up should remain the standard of care. C_LI

PURPOSEThe VISION study1 found that Lutetium-177 (177Lu)-PSMA-617 ("Lu-177") improved overall survival in metastatic castrate resistant prostate cancer (mCRPC). We assessed whether artificial intelligence enhanced PSMA imaging in mCRPC patients starting Lu-177 could identify those with better treatment outcomes. PATIENTS AND METHODSWe conducted a single site, tertiary center, retrospective cohort study in 51 consecutive mCRPC patients treated 2022-2024 with Lu-177. These patients had received most standard treatments, with disease progression. Planned treatment was Lu-177 every 6 weeks while continuing androgen deprivation therapy. Before starting treatment, PSMA images were analyzed for SUVmax and quantified tumor volume using artificial intelligence software (aPROMISE, Exinni Inc.). RESULTSFifty-one mCRPC patients were treated with Lu-177; 33 (65%) received 4 or more treatment cycles and these 33 had Kaplan-Meier median overall survival (OS) of 19.3 months and 23 (70%) surviving at 24 month data analysis. At first cycle Lu-177, these 33 had significantly more favorable levels of serum albumin, alkaline phosphatase, calcium, glucose, prostate specific antigen (PSA), ECOG performance status, and F18 PSMA imaging SUV-maximum values - reflecting PSMA "target expression". In a "protocol-eligibility" analysis, 30 of the 51 patients (59%) were considered "protocol-eligible" and 21 (41%) "protocol-ineligible" based on initial clinical parameters, as defined in Methods. "Protocol-eligible" patients had OS of 14.6 mo and 63% survival at 24 months. AI-enhanced F18 PSMA quantified imaging found "protocol-eligible" tumor volume in mL to be only 39% of the volume in "ineligible" patients. CONCLUSIONIn this cohort of mCRPC patients receiving Lu-177, pre-treatment AI-assisted F18 PSMA imaging finding higher PSMA SUV / lower tumor volume associated with the patients ability to have four or more treatment cycles, protocol eligibility, and better overall survival. KEY POINTSO_ST_ABSQuestionC_ST_ABSIn mCRPC patients initiating Lu-177 therapy, can AI-assisted F18 PSMA imaging identify patients who have better treatment outomes? Findings33 (65%) of a 51 consecutive patient mCRPC cohort were able to receive 4 or more cycles Lu-177. These patients had significantly more favorable serum albumin, alkaline phosphatase, calcium, glucose, PSA, performance status, and higher AI-PSMA scan SUV-maximum values, with a trend toward lower PSMA tumor volumes in mL. They had Kaplan-Meier median OS of 19.3 months and 70% survived at 24 months. AI-enhanced PSMA tumor volumes (mL) in "protocol eligible" patients were significantly lower - only 40% - than tumor volumes of "protocol ineligible" patients. MeaningIn this cohort of mCRPC patients receiving Lu-177, pre-treatment AI-assisted F18 PSMA imaging finding higher PSMA SUV / lower tumor volume associated with the patients ability to have four or more treatment cycles, protocol eligibility, and better overall survival.