Medical image translation with deep learning: Advances, datasets and perspectives.
Authors
Affiliations (7)
Affiliations (7)
- School of Software, Dalian University of Technology, Dalian 116621, China. Electronic address: [email protected].
- School of Software, Dalian University of Technology, Dalian 116621, China. Electronic address: [email protected].
- Institute of Research and Clinical Innovations, Neusoft Medical Systems Co., Ltd., Beijing, China. Electronic address: [email protected].
- School of Computing Science, University of Glasgow, Glasgow G12 8QQ, United Kingdom. Electronic address: [email protected].
- School of Computer Science, The University of Sydney, Sydney, NSW 2006, Australia. Electronic address: [email protected].
- Department of Radiology, General Hospital of Northern Theater Command, Shenyang 110840, China. Electronic address: [email protected].
- Guangdong-Hong Kong-Macao Joint Laboratory for Emotion Intelligence and Pervasive Computing, Artificial Intelligence Research Institute, Shenzhen MSU-BIT University, Shenzhen 518172, China; School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China. Electronic address: [email protected].
Abstract
Traditional medical image generation often lacks patient-specific clinical information, limiting its clinical utility despite enhancing downstream task performance. In contrast, medical image translation precisely converts images from one modality to another, preserving both anatomical structures and cross-modal features, thus enabling efficient and accurate modality transfer and offering unique advantages for model development and clinical practice. This paper reviews the latest advancements in deep learning(DL)-based medical image translation. Initially, it elaborates on the diverse tasks and practical applications of medical image translation. Subsequently, it provides an overview of fundamental models, including convolutional neural networks (CNNs), transformers, and state space models (SSMs). Additionally, it delves into generative models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Autoregressive Models (ARs), diffusion Models, and flow Models. Evaluation metrics for assessing translation quality are discussed, emphasizing their importance. Commonly used datasets in this field are also analyzed, highlighting their unique characteristics and applications. Looking ahead, the paper identifies future trends, challenges, and proposes research directions and solutions in medical image translation. It aims to serve as a valuable reference and inspiration for researchers, driving continued progress and innovation in this area.