BackgroundMedical professionals may increase diagnostic accuracy using multimodal medical image fusion techniques to peer inside organs and tissues.ObjectiveThis research work aims to propose a solution for diverse medical diagnostic challenges.MethodsWe propose a dual-purpose model. Initially, we developed a pair of images using the intensity, hue, and saturation (IHS) approach. Next, we applied non-subsampled shearlet transform (NSST) decomposition to these images to obtain the low-frequency and high-frequency coefficients. We then enhanced the structure and background details of the low-frequency coefficients using a novel structure feature modification technique. For the high-frequency coefficients, we utilized the layer-weighted pulse coupled neural network fusion technique to acquire complementary pixel-level information. Finally, we employed reversed NSST and IHS to generate the fused resulting image.ResultsThe proposed approach has been verified on 1350 image sets from two different diseases, Alzheimer's and glioma, across numerous imaging modalities. Our proposed method beats existing cutting-edge models, as proven by both qualitative and quantitative evaluations, and provides valuable information for medical diagnosis. In the majority of cases, our proposed method performed well in terms of entropy, structure similarity index, standard deviation, average distance, and average pixel intensity due to the careful selection of unique fusion strategies in our model. However, in a few cases, NSSTSIPCA performs better than our proposed work in terms of intensity variations (mean absolute error and average distance).ConclusionsThis research work utilized various fusion strategies in the NSST domain to efficiently enhance structural, anatomical, and spectral information.

Targeted ultrasound is commonly used to identify lesions characterized on magnetic resonance imaging (MRI) that were not recognized on initial mammography or ultrasound and is especially valuable for guiding percutaneous biopsies. Although artificial intelligence (AI) algorithms have been used to differentiate benign from malignant breast lesions on ultrasound, their application in classifying lesions on targeted ultrasound has not yet been studied. To evaluate the performance of AI-based software in predicting malignancy risk in breast lesions identified on targeted ultrasound. This was a retrospective, cross-sectional, single-center study that included patients with breast lesions identified on MRI who underwent targeted ultrasound and percutaneous ultrasound-guided biopsy. The ultrasound findings were analyzed using AI-based software and subsequently correlated with the pathological results. 334 lesions were evaluated, including 183 mass and 151 non-mass lesions. On histological analysis, there were 257 (76.9 %) benign lesions, and 77 (23.1 %) malignant. Both the AI software and radiologists demonstrated high sensitivity in predicting the malignancy risk of the lesions. The specificity was higher when evaluated by the radiologist using the AI software compared to the radiologist's evaluation alone (p < 0.001). All lesions classified as BI-RADS 2 or 3 on targeted ultrasound by the radiologist or the AI software (n = 72; 21.6 %) showed benign pathology results. The AI software, when integrated into the radiologist's evaluation, demonstrated high diagnostic accuracy and improved specificity for both mass and non-mass lesions on targeted ultrasound, supporting more accurate biopsy decisions and potentially reducing false positives without missing cancers.

Magnetic Resonance Imaging (MRI) is indispensable for modern diagnostics because of its detailed anatomical and functional information without the use of ionizing radiation. However, acquiring multiple imaging sequences - such as T1-weighted (T1w) and T2-weighted (T2w) scans - can prolong scan times, increase patient discomfort, and raise healthcare costs. In this study, we propose an unsupervised framework based on a contrast-sensitive domain translation network with adaptive feature normalization to translate unpaired T2w MRI images into clinically acceptable T1w images. Our method employs adversarial training, along with cycle consistency, identity, and attention-guided loss functions. These components ensure that the generated images not only preserve essential anatomical details but also exhibit high visual fidelity compared to ground truth T1w images. Quantitative evaluation on a publicly available MRI dataset yielded a mean Peak Signal-to-Noise Ratio (PSNR) of 22.403 dB, a mean Structural Similarity Index (SSIM) of 0.775, Root Mean Squared Error (RMSE) of 0.078, and Mean Absolute Error (MAE) of 0.036. Additional analysis of pixel intensity and grayscale distributions further supported the consistency between the generated and ground truth images. Qualitative assessment included visual comparison to assess perceptual fidelity. These promising results suggest that a contrast-sensitive domain translation network with an adaptive feature normalization framework can effectively generate realistic T1w images from T2w inputs, potentially reducing the need for acquiring multiple sequences and thereby streamlining MRI protocols.

Early detection of steatotic liver disease (SLD) is critically important. In clinical practice, hepatic steatosis is frequently diagnosed using computed tomography (CT) performed for unrelated clinical indications. An equation for estimating magnetic resonance proton density fat fraction (MR-PDFF) using liver attenuation on non-contrast CT exists, but no equivalent equation exists for post-contrast CT. We sought to (1) determine whether an automated workflow can accurately measure liver attenuation, (2) validate previously identified optimal thresholds for liver or liver-spleen attenuation in post-contrast studies, and (3) develop a method for estimating MR-PDFF (FF) on post-contrast CT. The fully automated TotalSegmentator 'total' machine learning model was used to segment 3D liver and spleen from non-contrast and post-contrast CT scans. Mean attenuation was extracted from liver (L) and spleen (S) volumes and from manually placed regions of interest (ROIs) in multi-phase CT scans of two cohorts: derivation (n = 1740) and external validation (n = 1044). Non-linear regression was used to determine the optimal coefficients for three phase-specific (arterial, venous, delayed) increasing exponential decay equations relating post-contrast L to non-contrast L. MR-PDFF was estimated from non-contrast CT and used as the reference standard. The mean attenuation for manual ROIs versus automated volumes were nearly perfectly correlated for both liver and spleen (r > .96, p < .001). For moderate-to-severe steatosis (L < 40 HU), the density of the liver (L) alone was a better classifier than either liver-spleen difference (L-S) or ratio (L/S) on post-contrast CTs. Fat fraction calculated using a corrected post-contrast liver attenuation measure agreed with non-contrast FF > 15% in both the derivation and external validation cohort, with AUROC between 0.92 and 0.97 on arterial, venous, and delayed phases. Automated volumetric mean attenuation of liver and spleen can be used instead of manually placed ROIs for liver fat assessments. Liver attenuation alone in post-contrast phases can be used to assess the presence of moderate-to-severe hepatic steatosis. Correction equations for liver attenuation on post-contrast phase CT scans enable reasonable quantification of liver steatosis, providing potential opportunities for utilizing clinical scans to develop large scale screening or studies in SLD.

Identifying disease risk and detecting disease before clinical symptoms appear are essential for early intervention and improving patient outcomes. In this context, the integration of medical imaging in a clinical workflow offers a unique advantage by capturing detailed structural and functional information. Unlike non-image data, such as lifestyle, sociodemographic, or prior medical conditions, which often rely on self-reported information susceptible to recall biases and subjective perceptions, imaging offers more objective and reliable insights. Although the use of medical imaging in artificial intelligence (AI)-driven risk assessment is growing, its full potential remains underutilized. In this work, we demonstrate how imaging can be integrated into routine screening workflows, in particular by taking advantage of neck-to-knee whole-body magnetic resonance imaging (MRI) data available in the large prospective study UK Biobank. Our analysis focuses on three-year risk assessment for a broad spectrum of diseases, including cardiovascular, digestive, metabolic, inflammatory, degenerative, and oncologic conditions. We evaluate AI-based pipelines for processing whole-body MRI and demonstrate that using image-derived radiomics features provides the best prediction performance, interpretability, and integration capability with non-image data.

Radiologists currently have very limited and time-consuming options to annotate findings on the images and are mostly limited to arrows, calipers and lines to annotate any type of findings on most PACS systems. We propose a framework placing encoded, transferable, highly contextual structured text annotations directly on PACS images indicating the type of lesion, level of suspicion, location, lesion measurement, and TNM status for malignant lesions, along with automated integration of this information into the radiology report. This approach offers a one-stop solution to generate radiology reports that are easily understood by other radiologists, patient care providers, patients, and machines while reducing the effort needed to dictate a detailed radiology report and minimizing speech recognition errors. It also provides a framework for automated generation of large volume high quality annotated data sets for machine learning algorithms from daily work of radiologists. Enabling voice dictation of these contextual annotations directly into PACS similar to voice enabled Google search will further enhance the user experience. Wider adaptation of contextualized structured annotations in the future can facilitate studies understanding the temporal evolution of different tumor lesions across multiple lines of treatment and early detection of asynchronous response/areas of treatment failure. We present a futuristic vision, and solution with the potential to transform clinical work and research in oncologic imaging.

An effective diagnosis system and suitable treatment planning require the precise segmentation of thyroid nodules in ultrasound imaging. The advancement of imaging technologies has not resolved traditional imaging challenges, which include noise issues, limited contrast, and dependency on operator choices, thus highlighting the need for automated, reliable solutions. The researchers developed TATHA, an innovative deep learning architecture dedicated to improving thyroid ultrasound image segmentation accuracy. The model is evaluated using the digital database of thyroid ultrasound images, which includes 99 cases across three subsets containing 134 labelled images for training, validation, and testing. It incorporates data pre-treatment procedures that reduce speckle noise and enhance contrast, while edge detection provides high-quality input for segmentation. TATHA outperforms U-Net, PSPNet, and Vision Transformers across various datasets and cross-validation folds, achieving superior Dice scores, accuracy, and AUC results. The distributed thyroid segmentation framework generates reliable predictions by combining results from multiple feature extraction units. The findings confirm that these advancements make TATHA an essential tool for clinicians and researchers in thyroid imaging and clinical applications.

Ulcerative colitis (UC) is a chronic inflammatory disorder necessitating precise severity stratification to facilitate optimal therapeutic interventions. This study harnesses a triple-pronged deep learning methodology-including multimodal inference pipelines that eliminate domain-specific training, few-shot meta-learning, and Vision Transformer (ViT)-based ensembling-to classify UC severity within the HyperKvasir dataset. We systematically evaluate multiple vision transformer architectures, discovering that a Swin-Base model achieves an accuracy of 90%, while a soft-voting ensemble of diverse ViT backbones boosts performance to 93%. In parallel, we leverage multimodal pre-trained frameworks (e.g., CLIP, BLIP, FLAVA) integrated with conventional machine learning algorithms, yielding an accuracy of 83%. To address limited annotated data, we deploy few-shot meta-learning approaches (e.g., Matching Networks), attaining 83% accuracy in a 5-shot context. Furthermore, interpretability is enhanced via SHapley Additive exPlanations (SHAP), which interpret both local and global model behaviors, thereby fostering clinical trust in the model's inferences. These findings underscore the potential of contemporary representation learning and ensemble strategies for robust UC severity classification, highlighting the pivotal role of model transparency in facilitating medical image analysis.

Diffusion MRI (dMRI) is an advanced imaging technique that enables in-vivo tracking of white matter fiber tracts and estimates the underlying cellular microstructure of brain tissues. Groupwise registration of dMRI data from multiple individuals is an important task for brain template creation and investigation of inter-subject brain variability. However, groupwise registration is a challenging task due to the uniqueness of dMRI data that include multi-dimensional, orientation-dependent signals that describe not only the strength but also the orientation of water diffusion in brain tissues. Deep learning approaches have shown successful performance in standard subject-to-subject dMRI registration. However, no deep learning methods have yet been proposed for groupwise dMRI registration. . In this work, we propose Deep Diffusion MRI Template (DDTemplate), which is a novel deep-learning-based method building upon the popular VoxelMorph framework to take into account dMRI fiber tract information. DDTemplate enables joint usage of whole-brain tissue microstructure and tract-specific fiber orientation information to ensure alignment of white matter fiber tracts and whole brain anatomical structures. We propose a novel deep learning framework that simultaneously trains a groupwise dMRI registration network and generates a population brain template. During inference, the trained model can be applied to register unseen subjects to the learned template. We compare DDTemplate with several state-of-the-art registration methods and demonstrate superior performance on dMRI data from multiple cohorts (adolescents, young adults, and elderly adults) acquired from different scanners. Furthermore, as a testbed task, we perform a between-population analysis to investigate sex differences in the brain, using the popular Tract-Based Spatial Statistics (TBSS) method that relies on groupwise dMRI registration. We find that using DDTemplate can increase the sensitivity in population difference detection, showing the potential of our method's utility in real neuroscientific applications.

Artificial Intelligence (AI), and particularly deep learning (DL), has shown great promise to revolutionize healthcare. However, clinical translation is often hindered by demanding hardware requirements. In this study, we assess the effectiveness of optimization techniques for DL models in healthcare applications, targeting varying AI workloads across the domains of radiology, histopathology, and medical RGB imaging, while evaluating across hardware configurations. The assessed AI workloads focus on both segmentation and classification workloads, by virtue of brain extraction in Magnetic Resonance Imaging (MRI), colorectal cancer delineation in Hematoxylin & Eosin (H&E) stained digitized tissue sections, and diabetic foot ulcer classification in RGB images. We quantitatively evaluate model performance in terms of model runtime during inference (including speedup, latency, and memory usage) and model utility on unseen data. Our results demonstrate that optimization techniques can substantially improve model runtime, without compromising model utility. These findings suggest that optimization techniques can facilitate the clinical translation of AI models in low-resource environments, making them more practical for real-world healthcare applications even in underserved regions.