Temporally consistent longitudinal brain tumor segmentation using a temporal spatial transformer network.
Authors
Affiliations (5)
Affiliations (5)
- School of Computing Science and Engineering, Galgotias University, Greater Noida, Uttar Pradesh, 203201, India.
- Department of Communication Technology and Network, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia (UPM), Serdang, Selangor, 43400, Malaysia.
- Department of Mathematics, Faculty of Mathematics and Natural Sciences, University of Jember, Jember, Indonesia.
- Manipal Institute of Technology Bengaluru, Manipal Academy of Higher Education, Manipal, India. [email protected].
- Department of Computer Science and Engineering, KG Reddy College of Engineering and Technology, Hyderabad, Telangana, 501504, India.
Abstract
Proper monitoring of tumor progression and evaluation of treatment responses highly depend on longitudinal brain tumor segmentation from MRI data. Current deep learning methodologies have mainly concentrated on analyzing single-time-point images, which restricts the ability to incorporate temporal dynamics during the segmentation process. The study proposes a new approach called Temporal-Spatial Transformer Network (TST-Net), which can be used for longitudinal brain tumor segmentation. The proposed methodology involves a temporal attention module to integrate the progression-aware information across temporally successive MRI images and a spatial attention module to improve tumor detection. Preprocessing of BraTS longitudinal MRI data and subsequent training of TST-Net on pre-aligned datasets were done in an end-to-end fashion. Evaluation was done by Dice similarity coefficients and compared with current methods. TST-Net proved to be more effective than previous solutions, obtaining 86.0%, 88.0%, and 91.0% Dice scores for improving tumor segmentation, tumor core, and overall tumor segmentation, respectively. This study reveals that incorporating the temporal dimension along with spatial attention leads to increased accuracy of brain tumor segmentation. Incorporating temporal information and applying spatial attention can effectively decrease inconsistencies between follow-up images and contribute to improved tumor area detection. TST-Net can serve as an important tool for clinical applications. TST-Net presents an efficient solution to brain tumor segmentation due to its ability to combine both spatial and temporal attention.