M<sup>4</sup>4TMD: A Multimodal, Multi-task Deep Learning Framework for Comprehensive Assessment of TMD-Related Abnormalities.
Authors
Affiliations (8)
Affiliations (8)
- Center for Plastic & Reconstructive Surgery, Department of Stomatology, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China.
- Department of General Dentistry, The Second Affiliated Hospital of Zhejiang University, Hangzhou, Zhejiang, China.
- College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China; Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, China.
- Weill Cornell Medicine, Cornell University, New York, USA. Electronic address: [email protected].
- School of Computing, National University of Singapore, Singapore, Singapore.
- Department of General Dentistry, The Second Affiliated Hospital of Zhejiang University, Hangzhou, Zhejiang, China. Electronic address: [email protected].
- College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China.
- Center for Plastic & Reconstructive Surgery, Department of Stomatology, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China. Electronic address: [email protected].
Abstract
Existing deep learning (DL) approaches for assessing temporomandibular disorders (TMD) are limited by underutilization of magnetic resonance imaging (MRI) in some tasks, a narrow focus on single-task detection, and predominant reliance on unimodal data. This study proposes a multimodal DL framework to address these issues. We collected 12,690 MRI slices and clinical data from 765 participants (1,410 temporomandibular joints), with each joint annotated for degenerative joint disease (DJD), anterior disc displacement (ADD), and effusion. We developed M<sup>4</sup>4TMD, utilizing multimodal data including multi-sequence and multi-slice MRI with clinical data, for concurrent assessment of DJD, ADD, and effusion. Performance was benchmarked against three recent DL methods and four clinicians with varying expertise across internal, temporal, and external test sets; assessments included generalization and visual interpretability experiments. Built upon ResNet50, M<sup>4</sup>4TMD exhibited superior internal test performance (ROC-AUC: 0.831, 0.913, and 0.961), surpassing prior methods. The accuracy of M<sup>4</sup>4TMD for three abnormalities was superior to that of junior dentists and comparable to that of two senior dentists (10 and 20 years experience): DJD (74.9% vs. 74.9%/72.5%; P > 0.05), ADD (78.2% vs. 71.1%/75.8%; P > 0.05), and effusion (90.5% vs. 88.6%/79.6%; P < 0.05). Strong robustness and interpretability were validated through generalization and visual interpretability experiments. The M<sup>4</sup>4TMD framework enables concurrent assessment of TMD-related abnormalities by integrating multimodal MRI and clinical data, exhibiting assessment performance comparable to senior dentists and demonstrating excellent robustness. The M<sup>4</sup>4TMD framework represents a critical step toward advancing DL-based TMD diagnosis in clinical practice.