seg2med: a bridge from artificial anatomy to multimodal medical images.
Authors
Affiliations (7)
Affiliations (7)
- Medical Faculty Mannheim, Universitätsmedizin Mannheim Institute for Intelligent Systems in Medicine, Theodor-Kutzer-Ufer 1-3, Mannheim, 68167, GERMANY.
- Department of Biomedical Engineering, Optical Bioimaging Laboratory, 1 Engineering Drive 2, Singapore, 117576, SINGAPORE.
- Lehrstuhl für Informatik 5 (Mustererkennung), Friedrich-Alexander-Universität Erlangen-Nürnberg Technische Fakultät, Martensstr. 3, Erlangen, Erlangen, Bayern, 91058, GERMANY.
- Universitat Heidelberg Computer Assisted Clinical Medicine, Theodor-Kutzer-Ufer 1-3, Mannheim, BW, 68167, GERMANY.
- Department of Radiology, University Hospital Mannheim, Department of Radiology, Theodor-Kutzer-Ufer 1-3, Mannheim, 68187, GERMANY.
- Department of Radiology, University Hospital Mannheim, Department of Radiology, Theodor-Kutzer-Ufer 1-3, Mannheim, 68167, GERMANY.
- Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, Mannheim, Select, 68167, GERMANY.
Abstract
We present seg2med (segmentation-to-medical images), a modular framework for anatomy-driven multimodal medical image synthesis. The system integrates three components to enable high-fidelity, cross-modality generation of CT and MR images based on structured anatomical priors.
Approach. First, anatomical maps are independently derived from three sources: real patient data, XCAT digital phantoms, and anatomies-synthetic subjects created by combining organs from multiple patients. Second, we introduce PhysioSynth, a modality-specific simulator that converts anatomical masks into imaging-like prior volumes using tissue-dependent parameters (e.g., HU, T1, T2, ρ) and modality-specific signal models. It supports simulation of CT and multiple MR sequences, including GRE, SPACE, and VIBE. Third, the synthesized anatomical priors are used to train 2-channel conditional denoising diffusion probabilistic models (DDPMs), which take the anatomical prior as a structural condition alongside the noisy image, enabling it to generate high-quality, structurally aligned images within its modality.
Main results. The framework achieves a Structural Similarity Index Measure (SSIM) of 0.94 ± 0.02 for CT and 0.82 ± 0.12 for MR images compared to real patient data, and 0.78 ± 0.04 FSIM for simulated CT from XCAT. The generative quality is further supported by a Frechet Inception Distance (FID) of 3.62 for CT synthesis. In modality conversion tasks, seg2med attains SSIM scores of 0.91 ± 0.03 (MR→CT) and 0.77 ± 0.04 (CT→MR).
Significance. In anatomical fidelity evaluation, synthetic CT images achieve a mean Dice coefficient exceeding 0.90 for 11 key abdominal organs, and over 0.80 for 34 of 59 total organs. These results underscore seg2med's utility in cross-modality image synthesis, dataset augmentation, and anatomy-aware AI development in medical imaging.