Large Language Model Ensemble for Automated TNM Staging from Radiology Reports.
Authors
Affiliations (6)
Affiliations (6)
- Institute of Information Systems and Applications, National Tsing Hua University.
- Department of Computer Science and Engineering, National Chung Hsing University.
- Department of Computer Science and Information Engineering, Asia University.
- Institute of Library, Information and Media Science, University of Tsukuba.
- Graduate Institute of Data Science, Taipei Medical University, No.301, Yuantong Rd., Zhonghe Dist., 235, New Taipei City, Taiwan.
- Clinical Big Data Research Center, Taipei Medical University Hospital, Taipei City, Taiwan.
Abstract
Accurate TNM staging from lung cancer radiology reports is crucial for treatment planning and prognosis assessment. Manual staging processes are time-consuming and subject to inter-observer variability. Large language models (LLMs) offer opportunities to automate TNM staging with enhanced interpretability and clinical reasoning. We developed two complementary systems for automated TNM staging from English radiology reports. System I employs GPT-4o with reasoning-based few-shot learning and multi-step voting. System II integrates multiple LLMs (GPT-4o and Gemini-2) using DSPy framework with MIPROv2 optimization. In NTCIR-18 RadNLP 2024 English main task, our approaches achieved first (joint accuracy: 0.6543) and second place (joint accuracy: 0.6296), demonstrating superior performance in T, N, and M classification with accuracies of 0.7037/0.9136/0.8889 and 0.7284/0.9383/0.8395, respectively. Source code freely available at https://github.com/nlptmu/multi-expert-tnm-staging under MIT license. An archival snapshot of the version used in this study is deposited on Zenodo at https://doi.org/10.5281/zenodo.20338561. Implemented in Python 3.12+ with PyTorch 2.6 and DSPY 3.0, supporting Linux. Supplementary data are available at Bioinformatics online.