Deformable registration is essential in medical image analysis, especially for handling various multi- and mono-modal registration tasks in neuroimaging. Existing studies lack exploration of brain MR-CT registration, and face challenges in both accuracy and efficiency improvements of learning-based methods. To enlarge the practice of multi-modal registration in brain, we present SR-Reg, a new benchmark dataset comprising 180 volumetric paired MR-CT images and annotated anatomical regions. Building on this foundation, we introduce MambaMorph, a novel deformable registration network based on an efficient state space model Mamba for global feature learning, with a fine-grained feature extractor for low-level embedding. Experimental results demonstrate that MambaMorph surpasses advanced ConvNet-based and Transformer-based networks across several multi- and mono-modal tasks, showcasing impressive enhancements of efficacy and efficiency. Code and dataset are available at https://github.com/mileswyn/MambaMorph.

Accurate adult age estimation (AAE) is critical for forensic and anthropological applications, yet traditional methods relying on bone mineral density (BMD) face significant challenges due to biological variability and methodological limitations. This study aims to develop an end-to-end Deep Learning (DL) based pipeline for automated AAE using BMD from proximal femoral CT scans. The main objectives are to construct a large-scale dataset of 5151 CT scans from real-world clinical and cadaver cohorts, fine-tune the Segment Anything Model (SAM) for accurate femoral bone segmentation, and evaluate multiple convolutional neural networks (CNNs) for precise age estimation based on segmented BMD data. Model performance was assessed through cross-validation, internal clinical testing, and external post-mortem validation. SAM achieved excellent segmentation performance with a Dice coefficient of 0.928 and an average intersection over union (mIoU) of 0.869. The CNN models achieved an average mean absolute error (MAE) of 5.20 years in cross-validation (male: 5.72; female: 4.51), which improved to 4.98 years in the independent clinical test set (male: 5.32; female: 4.56). External validation on the post-mortem dataset revealed an MAE of 6.91 years, with 6.97 for males and 6.69 for females. Ensemble learning further improved accuracy, reducing MAE to 4.78 years (male: 5.12; female: 4.35) in the internal test set, and 6.58 years (male: 6.64; female: 6.37) in the external validation set. These findings highlight the feasibility of dl-driven AAE and its potential for forensic applications, offering a fully automated framework for robust age estimation.

To develop a positron emission tomography/computed tomography (PET/CT)-based radiomics model for predicting programmed cell death ligand 1 (PD-L1) expression in non-small cell lung cancer (NSCLC) patients and estimating progression-free survival (PFS) and overall survival (OS) in PD-L1-positive patients undergoing first-line immunotherapy. We retrospectively analysed 143 NSCLC patients who underwent pretreatment ¹⁸F-fluorodeoxyglucose (¹⁸F-FDG) PET/CT scans, of whom 86 were PD-L1-positive. Clinical data collected included gender, age, smoking history, Tumor-Node-Metastases (TNM) staging system, pathologic types, laboratory parameters, and PET metabolic parameters. Four machine learning algorithms-Bayes, logistic, random forest, and Supportsupport vector machine (SVM)-were used to build models. The predictive performance was validated using receiver operating characteristic (ROC) curves. Univariate and multivariate Cox analyses identified independent predictors of OS and PFS in PD-L1-positive expression patients undergoing immunotherapy, and a nomogram was created to predict OS. A total of 20 models were built for predicting PD-L1 expression. The clinical combined PET/CT radiomics model based on the SVM algorithm performed best (area under curve for training and test sets: 0.914 and 0.877, respectively). The Cox analyses showed that smoking history independently predicted PFS. SUVmean, monocyte percentage and white blood cell count were independent predictors of OS, and the nomogram was created to predict 1-year, 2-year, and 3-year OS based on these three factors. We developed PET/CT-based machine learning models to help predict PD-L1 expression in NSCLC patients and identified independent predictors of PFS and OS in PD-L1-positive patients receiving immunotherapy, thereby aiding precision treatment.

Embolic protection devices (EPDs) used during carotid artery stenting (CAS) are crucial in reducing ischemic complications. Although minimizing the filter-type EPD movement is considered important, limited research has demonstrated this practice. We used an artificial intelligence (AI)-based device recognition technology to investigate the correlation between filter movements and ischemic complications. We retrospectively studied 28 consecutive patients who underwent CAS using FilterWire EZ (Boston Scientific, Marlborough, MA, USA) from April 2022 to September 2023. Clinical data, procedural videos, and postoperative magnetic resonance imaging were collected. An AI-based device detection function in the Neuro-Vascular Assist (iMed Technologies, Tokyo, Japan) was used to quantify the filter movement. Multivariate proportional odds model analysis was performed to explore the correlations between postoperative diffusion-weighted imaging (DWI) hyperintense lesions and potential ischemic risk factors, including filter movement. In total, 23 patients had sufficient information and were eligible for quantitative analysis. Fourteen patients (60.9 %) showed postoperative DWI hyperintense lesions. Multivariate analysis revealed significant associations between filter movement distance (odds ratio, 1.01; 95 % confidence interval, 1.00-1.02; p = 0.003) and high-intensity signals in time-of-flight magnetic resonance angiography with DWI hyperintense lesions. Age, symptomatic status, and operative time were not significantly correlated. Increased filter movement during CAS was correlated with a higher incidence of postoperative DWI hyperintense lesions. AI-based quantitative evaluation of endovascular techniques may enable demonstration of previously unproven recommendations. To the best of our knowledge, this is the first study to use an AI system for quantitative evaluation to address real-world clinical issues.

It is a challenging task to use ultrasound for bone imaging, as the bone tissue has a complex structure with high acoustic impedance and speed-of-sound (SOS). Recently, full waveform inversion (FWI) has shown promising imaging for musculoskeletal tissues. However, the FWI showed a limited ability and tended to produce artifacts in bone imaging because the inversion process would be more easily trapped in local minimum for bone tissue with a large discrepancy in SOS distribution between bony and soft tissues. In addition, the application of FWI required a high computational burden and relatively long iterations. The objective of this study was to achieve high-resolution ultrasonic imaging of bone using a deep learning-based FWI approach. In this paper, we proposed a novel network named CEDD-Unet. The CEDD-Unet adopts a Dual-Decoder architecture, with the first decoder tasked with reconstructing the SOS model, and the second decoder tasked with finding the main boundaries between bony and soft tissues. To effectively capture multi-scale spatial-temporal features from ultrasound radio frequency (RF) signals, we integrated a Convolutional LSTM (ConvLSTM) module. Additionally, an Efficient Multi-scale Attention (EMA) module was incorporated into the encoder to enhance feature representation and improve reconstruction accuracy. Using the ultrasonic imaging modality with a ring array transducer, the performance of CEDD-Unet was tested on the SOS model datasets from human bones (noted as Dataset1) and mouse bones (noted as Dataset2), and compared with three classic reconstruction architectures (Unet, Unet++, and Att-Unet), four state-of-the-art architecture (InversionNet, DD-Net, UPFWI, and DEFE-Unet). Experiments showed that CEDD-Unet outperforms all competing methods, achieving the lowest MAE of 23.30 on Dataset1 and 25.29 on Dataset2, the highest SSIM of 0.9702 on Dataset1 and 0.9550 on Dataset2, and the highest PSNR of 30.60 dB on Dataset1 and 32.87 dB on Dataset2. Our method demonstrated superior reconstruction quality, with clearer bone boundaries, reduced artifacts, and improved consistency with ground truth. Moreover, CEDD-Unet surpasses traditional FWI by producing sharper skeletal SOS reconstructions, reducing computational cost, and eliminating the reliance for an initial model. Ablation studies further confirm the effectiveness of each network component. The results suggest that CEDD-Unet is a promising deep learning-based FWI method for high-resolution bone imaging, with the potential to reconstruct accurate and sharp-edged skeletal SOS models.

Quality control (QC) of structures segmentation in volumetric medical images is important for identifying segmentation errors in clinical practice and for facilitating model development by enhancing network performance in semi-supervised and active learning scenarios. This paper introduces SegQC, a novel framework for segmentation quality estimation and segmentation error detection. SegQC computes an estimate measure of the quality of a segmentation in volumetric scans and in their individual slices and identifies possible segmentation error regions within a slice. The key components of SegQC include: 1) SegQCNet, a deep network that inputs a scan and its segmentation mask and outputs segmentation error probabilities for each voxel in the scan; 2) three new segmentation quality metrics computed from the segmentation error probabilities; 3) a new method for detecting possible segmentation errors in scan slices computed from the segmentation error probabilities. We introduce a novel evaluation scheme to measure segmentation error discrepancies based on an expert radiologist's corrections of automatically produced segmentations that yields smaller observer variability and is closer to actual segmentation errors. We demonstrate SegQC on three fetal structures in 198 fetal MRI scans - fetal brain, fetal body and the placenta. To assess the benefits of SegQC, we compare it to the unsupervised Test Time Augmentation (TTA)-based QC and to supervised autoencoder (AE)-based QC. Our studies indicate that SegQC outperforms TTA-based quality estimation for whole scans and individual slices in terms of Pearson correlation and MAE for fetal body and fetal brain structures segmentation as well as for volumetric overlap metrics estimation of the placenta structure. Compared to both unsupervised TTA and supervised AE methods, SegQC achieves lower MAE for both 3D and 2D Dice estimates and higher Pearson correlation for volumetric Dice. Our segmentation error detection method achieved recall and precision rates of 0.77 and 0.48 for fetal body, and 0.74 and 0.55 for fetal brain segmentation error detection, respectively. Ranking derived from metrics estimation surpasses rankings based on entropy and sum for TTA and SegQCNet estimations, respectively. SegQC provides high-quality metrics estimation for both 2D and 3D medical images as well as error localization within slices, offering important improvements to segmentation QC.

Optimize deep learning-based vertebrae segmentation in longitudinal CT scans of multiple myeloma patients using structural uncertainty analysis. Retrospective CT scans from 474 multiple myeloma patients were divided into train (179 patients, 349 scans, 2005-2011) and test cohort (295 patients, 671 scans, 2012-2020). An enhanced segmentation pipeline was developed on the train cohort. It integrated vertebrae segmentation using an open-source deep learning method (Payer's) with a post-hoc structural uncertainty analysis. This analysis identified inconsistencies, automatically correcting them or flagging uncertain regions for human review. Segmentation quality was assessed through vertebral shape analysis using topology. Metrics included 'identification rate', 'longitudinal vertebral match rate', 'success rate' and 'series success rate' and evaluated across age/sex subgroups. Statistical analysis included McNemar and Wilcoxon signed-rank tests, with p < 0.05 indicating significant improvement. Payer's method achieved an identification rate of 95.8% and success rate of 86.7%. The proposed pipeline automatically improved these metrics to 98.8% and 96.0%, respectively (p < 0.001). Additionally, 3.6% of scans were marked for human inspection, increasing the success rate from 96.0% to 98.8% (p < 0.001). The vertebral match rate increased from 97.0% to 99.7% (p < 0.001), and the series success rate from 80.0% to 95.4% (p < 0.001). Subgroup analysis showed more consistent performance across age and sex groups. The proposed pipeline significantly outperforms Payer's method, enhancing segmentation accuracy and reducing longitudinal matching errors while minimizing evaluation workload. Its uncertainty analysis ensures robust performance, making it a valuable tool for longitudinal studies in multiple myeloma.

In this paper, we introduce the Visual Query-based task of Video Clip Localization (VQ-VCL) for medical video understanding. Specifically, we aim to retrieve a video clip containing frames similar to a given exemplar frame from a given input video. To solve the task, we propose a novel visual query-based video clip localization model called TIER-LOC. TIER-LOC is designed to improve video clip retrieval, especially in fine-grained videos by extracting features from different levels, i.e., coarse to fine-grained, referred to as TIERS. The aim is to utilize multi-Tier features for detecting subtle differences, and adapting to scale or resolution variations, leading to improved video-clip retrieval. TIER-LOC has three main components: (1) a Multi-Tier Spatio-Temporal Transformer to fuse spatio-temporal features extracted from multiple Tiers of video frames with features from multiple Tiers of the visual query enabling better video understanding. (2) a Multi-Tier, Dual Anchor Contrastive Loss to deal with real-world annotation noise which can be notable at event boundaries and in videos featuring highly similar objects. (3) a Temporal Uncertainty-Aware Localization Loss designed to reduce the model sensitivity to imprecise event boundary. This is achieved by relaxing hard boundary constraints thus allowing the model to learn underlying class patterns and not be influenced by individual noisy samples. To demonstrate the efficacy of TIER-LOC, we evaluate it on two ultrasound video datasets and an open-source egocentric video dataset. First, we develop a sonographer workflow assistive task model to detect standard-frame clips in fetal ultrasound heart sweeps. Second, we assess our model's performance in retrieving standard-frame clips for detecting fetal anomalies in routine ultrasound scans, using the large-scale PULSE dataset. Lastly, we test our model's performance on an open-source computer vision video dataset by creating a VQ-VCL fine-grained video dataset based on the Ego4D dataset. Our model outperforms the best-performing state-of-the-art model by 7%, 4%, and 4% on the three video datasets, respectively.

This study aimed to develop a computed tomography (CT)-based multi-organ segmentation model for delineating organs-at-risk (OARs) in pediatric upper abdominal tumors and evaluate its robustness across multiple datasets. In-house postoperative CTs from pediatric patients with renal tumors and neuroblastoma (n = 189) and a public dataset (n = 189) with CTs covering thoracoabdominal regions were used. Seventeen OARs were delineated: nine by clinicians (Type 1) and eight using TotalSegmentator (Type 2). Auto-segmentation models were trained using in-house (Model-PMC-UMCU) and a combined dataset of public data (Model-Combined). Performance was assessed with Dice Similarity Coefficient (DSC), 95 % Hausdorff Distance (HD95), and mean surface distance (MSD). Two clinicians rated clinical acceptability on a 5-point Likert scale across 15 patient contours. Model robustness was evaluated against sex, age, intravenous contrast, and tumor type. Model-PMC-UMCU achieved mean DSC values above 0.95 for five of nine OARs, while the spleen and heart ranged between 0.90 and 0.95. The stomach-bowel and pancreas exhibited DSC values below 0.90. Model-Combined demonstrated improved robustness across both datasets. Clinical evaluation revealed good usability, with both clinicians rating six of nine Type 1 OARs above four and six of eight Type 2 OARs above three. Significant performance differences were only found across age groups in both datasets, specifically in the left lung and pancreas. The 0-2 age group showed the lowest performance. A multi-organ segmentation model was developed, showcasing enhanced robustness when trained on combined datasets. This model is suitable for various OARs and can be applied to multiple datasets in clinical settings.

Medical image segmentation is a pivotal task within the realms of medical image analysis and computer vision. While current methods have shown promise in accurately segmenting major regions of interest, the precise segmentation of boundary areas remains challenging. In this study, we propose a novel network architecture named CTO, which combines Convolutional Neural Networks (CNNs), Vision Transformer (ViT) models, and explicit edge detection operators to tackle this challenge. CTO surpasses existing methods in terms of segmentation accuracy and strikes a better balance between accuracy and efficiency, without the need for additional data inputs or label injections. Specifically, CTO adheres to the canonical encoder-decoder network paradigm, with a dual-stream encoder network comprising a mainstream CNN stream for capturing local features and an auxiliary StitchViT stream for integrating long-range dependencies. Furthermore, to enhance the model's ability to learn boundary areas, we introduce a boundary-guided decoder network that employs binary boundary masks generated by dedicated edge detection operators to provide explicit guidance during the decoding process. We validate the performance of CTO through extensive experiments conducted on seven challenging medical image segmentation datasets, namely ISIC 2016, PH2, ISIC 2018, CoNIC, LiTS17, BraTS, and BTCV. Our experimental results unequivocally demonstrate that CTO achieves state-of-the-art accuracy on these datasets while maintaining competitive model complexity. The codes have been released at: CTO.