Accurate MR signal simulation, including microvascular structures and water diffusion, is crucial for MRI techniques like fMRI BOLD modeling and MR vascular Fingerprinting (MRF), which use susceptibility effects on MR signals for tissue characterization. However, integrating microvascular features and diffusion remains computationally challenging, limiting the accuracy of the estimates. Using advanced modeling and deep neural networks, we propose a novel simulation tool that efficiently accounts for susceptibility and diffusion effects. We used dimension reduction of magnetic field inhomogeneity matrices combined with deep learning methodology to accelerate the simulations while maintaining their accuracy. We validated our results through an in silico study against a reference method and in vivo MRF experiments. This approach accelerates MR signal generation by a factor of almost 13 000 compared to previously used simulation methods while preserving accuracy. The MR-WAVES method allows fast generation of MR signals accounting for microvascular structures and water-diffusion contribution.

Most current medical image segmentation models employ a unified feature modeling strategy for all target regions. However, they overlook the significant heterogeneity in lesion structure, boundary characteristics, and semantic texture, which frequently restricts their ability to accurately segment morphologically diverse lesions in complex imaging contexts, thereby reducing segmentation accuracy and robustness. To address this issue, we propose a brain-inspired segmentation framework named BrainNet, which adopts a tri-level backbone encoder-Brain Network-decoder architecture. Such an architecture enables globally guided, locally differentiated feature modeling. We further instantiate the framework with an attention-enhanced segmentation model, termed Att-BrainNet. In this model, a Thalamus Gating Module (TGM) dynamically selects and activates structurally identical but functionally diverse Encephalic Region Networks (ERNs) to collaboratively extract lesion-specific features. In addition, an S-F image enhancement module is incorporated to improve sensitivity to boundaries and fine structures. Meanwhile, multi-head self-attention is embedded in the encoder to strengthen global semantic modeling and regional coordination. Experiments conducted on two lung cancer CT segmentation datasets and the Synapse multi-organ dataset demonstrate that Att-BrainNet outperforms existing mainstream segmentation models in terms of both accuracy and generalization. Further ablation studies and mechanism visualizations confirm the effectiveness of the BrainNet architecture and the dynamic scheduling strategy. This research provides a novel structural paradigm for medical image segmentation and holds promise for extension to other complex segmentation scenarios.

Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are essential clinical cross-sectional imaging techniques for diagnosing complex conditions. However, large 3D datasets with annotations for deep learning are scarce. While methods like DINOv2 are encouraging for 2D image analysis, these methods have not been applied to 3D medical images. Furthermore, deep learning models often lack explainability due to their "black-box" nature. This study aims to extend 2D self-supervised models, specifically DINOv2, to 3D medical imaging while evaluating their potential for explainable outcomes. We introduce the Medical Slice Transformer (MST) framework to adapt 2D self-supervised models for 3D medical image analysis. MST combines a Transformer architecture with a 2D feature extractor, i.e., DINOv2. We evaluate its diagnostic performance against a 3D convolutional neural network (3D ResNet) across three clinical datasets: breast MRI (651 patients), chest CT (722 patients), and knee MRI (1199 patients). Both methods were tested for diagnosing breast cancer, predicting lung nodule dignity, and detecting meniscus tears. Diagnostic performance was assessed by calculating the Area Under the Receiver Operating Characteristic Curve (AUC). Explainability was evaluated through a radiologist's qualitative comparison of saliency maps based on slice and lesion correctness. P-values were calculated using Delong's test. MST achieved higher AUC values compared to ResNet across all three datasets: breast (0.94 ± 0.01 vs. 0.91 ± 0.02, P = 0.02), chest (0.95 ± 0.01 vs. 0.92 ± 0.02, P = 0.13), and knee (0.85 ± 0.04 vs. 0.69 ± 0.05, P = 0.001). Saliency maps were consistently more precise and anatomically correct for MST than for ResNet. Self-supervised 2D models like DINOv2 can be effectively adapted for 3D medical imaging using MST, offering enhanced diagnostic accuracy and explainability compared to convolutional neural networks.

Harmonic Motion Imaging (HMI) is an ultrasound elasticity imaging method that measures the mechanical properties of tissue using amplitude-modulated acoustic radiation force (AM-ARF). Multi-frequency HMI (MF-HMI) excites tissue at various AM frequencies simultaneously, allowing for image optimization without prior knowledge of inclusion size and stiffness. However, challenges remain in size estimation as inconsistent boundary effects result in different perceived sizes across AM frequencies. Herein, we developed an automated assessment method for tumor and focused ultrasound surgery (FUS) induced lesions using a transformer-based multi-modality neural network, HMINet, and further automated neoadjuvant chemotherapy (NACT) response prediction. HMINet was trained on 380 pairs of MF-HMI and B-mode images of phantoms and in vivo orthotopic breast cancer mice (4T1). Test datasets included phantoms (n = 32), in vivo 4T1 mice (n = 24), breast cancer patients (n = 20), FUS-induced lesions in ex vivo animal tissue and in vivo clinical settings with real-time inference, with average segmentation accuracy (Dice) of 0.91, 0.83, 0.80, and 0.81, respectively. HMINet outperformed state-of-the-art models; we also demonstrated the enhanced robustness of the multi-modality strategy over B-mode-only, both quantitatively through Dice scores and in terms of interpretation using saliency analysis. The contribution of AM frequency based on the number of salient pixels showed that the most significant AM frequencies are 800 and 200 Hz across clinical cases. We developed an automated, multimodality ultrasound-based tumor and FUS lesion assessment method, which facilitates the clinical translation of stiffness-based breast cancer treatment response prediction and real-time image-guided FUS therapy.

The integration of artificial intelligence in medical image classification has significantly advanced disease detection. However, traditional deep learning models face persistent challenges, including poor generalizability, high false-positive rates, and difficulties in distinguishing overlapping anatomical features, limiting their clinical utility. To address these limitations, this study proposes a hybrid framework combining Vision Transformers (ViT) and Perceiver IO, designed to enhance multi-disease classification accuracy. Vision Transformers leverage self-attention mechanisms to capture global dependencies in medical images, while Perceiver IO optimizes feature extraction for computational efficiency and precision. The framework is evaluated across three critical clinical domains: neurological disorders, including Stroke (tested on the Brain Stroke Prediction CT Scan Image Dataset) and Alzheimer's (analyzed via the Best Alzheimer MRI Dataset); skin diseases, covering Tinea (trained on the Skin Diseases Dataset) and Melanoma (augmented with dermoscopic images from the HAM10000/HAM10k dataset); and lung diseases, focusing on Lung Cancer (using the Lung Cancer Image Dataset) and Pneumonia (evaluated with the Pneumonia Dataset containing bacterial, viral, and normal X-ray cases). For neurological disorders, the model achieved 0.99 accuracy, 0.99 precision, 1.00 recall, 0.99 F1-score, demonstrating robust detection of structural brain abnormalities. In skin disease classification, it attained 0.95 accuracy, 0.93 precision, 0.97 recall, 0.95 F1-score, highlighting its ability to differentiate fine-grained textural patterns in lesions. For lung diseases, the framework achieved 0.98 accuracy, 0.97 precision, 1.00 recall, 0.98 F1-score, confirming its efficacy in identifying respiratory conditions. To bridge research and clinical practice, an AI-powered chatbot was developed for real-time analysis, enabling users to upload MRI, X-ray, or skin images for automated diagnosis with confidence scores and interpretable insights. This work represents the first application of ViT and Perceiver IO for these disease categories, outperforming conventional architectures in accuracy, computational efficiency, and clinical interpretability. The framework holds significant potential for early disease detection in healthcare settings, reducing diagnostic errors, and improving treatment outcomes for clinicians, radiologists, and patients. By addressing critical limitations of traditional models, such as overlapping feature confusion and false positives, this research advances the deployment of reliable AI tools in neurology, dermatology, and pulmonology.

Fine-grained visual classification is fundamental for medical image applications because it detects minor lesions. Diabetic retinopathy (DR) is a preventable cause of blindness, which requires exact and timely diagnosis to prevent vision damage. The challenges automated DR classification systems face include irregular lesions, uneven distributions between image classes, and inconsistent image quality that reduces diagnostic accuracy during early detection stages. Our solution to these problems includes MSCAS-Net (Multi-Scale Cross and Self-Attention Network), which uses the Swin Transformer as the backbone. It extracts features at three different resolutions (12 × 12, 24 × 24, 48 × 48), allowing it to detect subtle local features and global elements. This model uses self-attention mechanics to improve spatial connections between single scales and cross-attention to automatically match feature patterns across multiple scales, thereby developing a comprehensive information structure. The model becomes better at detecting significant lesions because of its dual mechanism, which focuses on both attention points. MSCAS-Net displays the best performance on APTOS and DDR and IDRID benchmarks by reaching accuracy levels of 93.8%, 89.80% and 86.70%, respectively. Through its algorithm, the model solves problems with imbalanced datasets and inconsistent image quality without needing data augmentation because it learns stable features. MSCAS-Net demonstrates a breakthrough in automated DR diagnostics since it combines high diagnostic precision with interpretable abilities to become an efficient AI-powered clinical decision support system. The presented research demonstrates how fine-grained visual classification methods benefit detecting and treating DR during its early stages.

Quantitative analysis of PET images in brain PET/CT relies on MRI-derived regions of interest (ROIs). However, the pairs of PET/CT and MR images are not always available, and their alignment is challenging if their acquisition times differ considerably. To address these problems, this study proposes a deep learning framework for translating CT of PET/CT to synthetic MR images (MR_SYN) and performing automated quantitative regional analysis using MR_SYN-derived segmentation. In this retrospective study, 139 subjects who underwent brain [¹⁸F]FBB PET/CT and T1-weighted MRI were included. A U-Net-like model was trained to translate CT images to MR_SYN; subsequently, a separate model was trained to segment MR_SYN into 95 regions. Regional and composite standardised uptake value ratio (SUVr) was calculated in [¹⁸F]FBB PET images using the acquired ROIs. For evaluation of MR_SYN, quantitative measurements including structural similarity index measure (SSIM) were employed, while for MR_SYN-based segmentation evaluation, Dice similarity coefficient (DSC) was calculated. Wilcoxon signed-rank test was performed for SUVrs computed using MR_SYN and ground-truth MR (MR_GT). Compared to MR_GT, the mean SSIM of MR_SYN was 0.974 ± 0.005. The MR_SYN-based segmentation achieved a mean DSC of 0.733 across 95 regions. No statistical significance (P > 0.05) was found for SUVr between the ROIs from MR_SYN and those from MR_GT, excluding the precuneus. We demonstrated a deep learning framework for automated regional brain analysis in PET/CT with MR_SYN. Our proposed framework can benefit patients who have difficulties in performing an MRI scan.

Magnetic resonance imaging (MRI) is a widely used radiological modality renowned for its radiation-free, comprehensive insights into the human body, facilitating medical diagnoses. However, the drawback of prolonged scan times hinders its accessibility. The k-space undersampling offers a solution, yet the resultant artifacts necessitate meticulous removal during image reconstruction. Although deep learning (DL) has proven effective for fast MRI image reconstruction, its broader applicability across various imaging scenarios has been constrained. Challenges include the high cost and privacy restrictions associated with acquiring large-scale, diverse training data, coupled with the inherent difficulty of addressing mismatches between training and target data in existing DL methodologies. Here, we present a novel Physics-Informed Synthetic data learning Framework for fast MRI, called PISF. PISF marks a breakthrough by enabling generalizable DL for multi-scenario MRI reconstruction through a single trained model. Our approach separates the reconstruction of a 2D image into many 1D basic problems, commencing with 1D data synthesis to facilitate generalization. We demonstrate that training DL models on synthetic data, coupled with enhanced learning techniques, yields in vivo MRI reconstructions comparable to or surpassing those of models trained on matched realistic datasets, reducing the reliance on real-world MRI data by up to 96 %. With a single trained model, our PISF supports the high-quality reconstruction under 4 sampling patterns, 5 anatomies, 6 contrasts, 5 vendors, and 7 centers, exhibiting remarkable generalizability. Its adaptability to 2 neuro and 2 cardiovascular patient populations has been validated through evaluations by 10 experienced medical professionals. In summary, PISF presents a feasible and cost-effective way to significantly boost the widespread adoption of DL in various fast MRI applications.

The reliance on computationally intensive U-Net and Transformer architectures significantly limits their accessibility in low-resource environments, creating a technological divide that hinders global healthcare equity, especially in medical diagnostics and treatment planning. This divide is most pronounced in low- and middle-income countries, primary care facilities, and conflict zones. We introduced MED-NCA, Neural Cellular Automata (NCA) based segmentation models characterized by their low parameter count, robust performance, and inherent quality control mechanisms. These features drastically lower the barriers to high-quality medical image analysis in resource-constrained settings, allowing the models to run efficiently on hardware as minimal as a Raspberry Pi or a smartphone. Building upon the foundation laid by MED-NCA, this paper extends its validation across eight distinct anatomies, including the hippocampus and prostate (MRI, 3D), liver and spleen (CT, 3D), heart and lung (X-ray, 2D), breast tumor (Ultrasound, 2D), and skin lesion (Image, 2D). Our comprehensive evaluation demonstrates the broad applicability and effectiveness of MED-NCA in various medical imaging contexts, matching the performance of two magnitudes larger UNet models. Additionally, we introduce NCA-VIS, a visualization tool that gives insight into the inference process of MED-NCA and allows users to test its robustness by applying various artifacts. This combination of efficiency, broad applicability, and enhanced interpretability makes MED-NCA a transformative solution for medical image analysis, fostering greater global healthcare equity by making advanced diagnostics accessible in even the most resource-limited environments.

Steatotic liver disease (SLD) is the most common liver disease worldwide, affecting 30% of the global population. It is strongly associated with the interplay of genetic and lifestyle-related risk factors. The genetic variant accounting for the largest fraction of SLD heritability is PNPLA3 I148M, which is carried by 23% of the western population and increases the risk of SLD two to three-fold. However, identification of variant carriers is not part of routine clinical care and prevents patients from receiving personalised care. We analysed MRI images and common genetic variants in PNPLA3, TM6SF2, MTARC1, HSD17B13 and GCKR from a cohort of 45 603 individuals from the UK Biobank. Proton density fat fraction (PDFF) maps were generated using a water-fat separation toolbox, applied to the magnitude and phase MRI data. The liver region was segmented using a U-Net model trained on 600 manually segmented ground truth images. The resulting liver masks and PDFF maps were subsequently used to calculate liver PDFF values. Individuals with (PDFF ≥ 5%) and without SLD (PDFF < 5%) were selected as the study cohort and used to train and test a Vision Transformer classification model with five-fold cross validation. We aimed to differentiate individuals who are homozygous for the PNPLA3 I148M variant from non-carriers, as evaluated by the area under the receiver operating characteristic curve (AUROC). To ensure a clear genetic contrast, all heterozygous individuals were excluded. To interpret our model, we generated attention maps that highlight the regions that are most predictive of the outcomes. Homozygosity for the PNPLA3 I148M variant demonstrated the best predictive performance among five variants with AUROC of 0.68 (95% CI: 0.64-0.73) in SLD patients and 0.57 (95% CI: 0.52-0.61) in non-SLD patients. The AUROCs for the other SNPs ranged from 0.54 to 0.57 in SLD patients and from 0.52 to 0.54 in non-SLD patients. The predictive performance was generally higher in SLD patients compared to non-SLD patients. Attention maps for PNPLA3 I148M carriers showed that fat deposition in regions adjacent to the hepatic vessels, near the liver hilum, plays an important role in predicting the presence of the I148M variant. Our study marks novel progress in the non-invasive detection of homozygosity for PNPLA3 I148M through the application of deep learning models on MRI images. Our findings suggest that PNPLA3 I148M might affect the liver fat distribution and could be used to predict the presence of PNPLA3 variants in patients with fatty liver. The findings of this research have the potential to be integrated into standard clinical practice, particularly when combined with clinical and biochemical data from other modalities to increase accuracy, enabling easier identification of at-risk individuals and facilitating the development of tailored interventions for PNPLA3 I148M-associated liver disease.