A new low-rank adaptation method for brain structure and metastasis segmentation via decoupled principal weight direction and magnitude.
Authors
Affiliations (5)
Affiliations (5)
- School of Mathematics, Physics and Information, Shaoxing University, 900 ChengNan Rd, Shaoxing, 312000, Zhejiang, China.
- The Affiliated Hospital of Shaoxing University, Shaoxing, 312000, Zhejiang, China.
- School of Mathematics, Physics and Information, Shaoxing University, 900 ChengNan Rd, Shaoxing, 312000, Zhejiang, China. [email protected].
- Cancer Center, Gamma Knife Treatment Center, Zhejiang Provincial People's Hospital, AffiliatedPeople's Hospital, Hangzhou Medical College, Hangzhou, 310014, Zhejiang, China. [email protected].
- School of Information Engineering, Huzhou University, Huzhou, 313000, Zhejiang, China.
Abstract
Deep learning techniques have become pivotal in medical image segmentation, but their success often relies on large, manually annotated datasets, which are expensive and labor-intensive to obtain. Additionally, different segmentation tasks frequently require retraining models from scratch, resulting in substantial computational costs. To address these limitations, we propose PDoRA, an innovative parameter-efficient fine-tuning method that leverages knowledge transfer from a pre-trained SwinUNETR model for a wide range of brain image segmentation tasks. PDoRA minimizes the reliance on extensive data annotation and computational resources by decomposing model weights into principal and residual weights. The principal weights are further divided into magnitude and direction, enabling independent fine-tuning to enhance the model's ability to capture task-specific features. The residual weights remain fixed and are later fused with the updated principal weights, ensuring model stability while enhancing performance. We evaluated PDoRA on three diverse medical image datasets for brain structure and metastasis segmentation. The results demonstrate that PDoRA consistently outperforms existing parameter-efficient fine-tuning methods, achieving superior segmentation accuracy and efficiency. Our code is available at https://github.com/Perfect199001/PDoRA/tree/main .