Comparison of three commercial AI tools for detection and malignancy assessment of incidental lung nodules.
Authors
Affiliations (6)
Affiliations (6)
- Department of Diagnostic and Interventional Radiology, Heidelberg University Hospital, Heidelberg, Germany.
- Translational Lung Research Center Heidelberg (TLRC), German Center for Lung Research (DZL), Heidelberg, Germany.
- Department of Diagnostic and Interventional Radiology with Nuclear Medicine, Thoraxklinik at University of Heidelberg, Heidelberg, Germany.
- Department of Radiology, Kütahya Health Sciences University, Kütahya, Türkiye.
- Department of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, Greifswald, Germany.
- Department of Nuclear Medicine, University Medicine Greifswald, Greifswald, Germany.
Abstract
Incidental pulmonary nodules are common on computed tomography (CT), and management typically relies on size and volume. Artificial intelligence (AI)-based tools show promise in nodule detection and risk assessment, but their clinical utility remains uncertain. This study aimed to evaluate the accuracy of AI-based software in detecting and predicting the malignancy of incidental pulmonary nodules. This retrospective study included patients selected from a cohort of 1,138 individuals who underwent chest CT between 2015 and 2024 and met the inclusion criteria. Patients were classified into benign and malignant groups. Malignancy was determined by histopathology or by at least 2 years of follow-up. Nodule location, size, and type were assessed using by both using radiology reports and AI tools. Three commercial tools (AI-I, AI-II, and AI-III) were assessed for nodule detection and malignancy risk prediction. Agreement was assessed using Cohen's kappa and the intraclass correlation coefficient (ICC), and diagnostic performance was evaluated using receiver operating characteristic analysis. Chest CT scans of 374 patients (mean age 66 ± 9 yr.; range 37-88 yr.; 231 males) with at least one solid or part-solid nodule were evaluated. AI-I and AI-II demonstrated excellent agreement with radiology reports for nodule localization (<i>κ</i> = 0.95, <i>p</i> < 0.001) and moderate agreement for nodule type (<i>κ</i> = 0.46, <i>p</i> < 0.001). Size assessment showed excellent agreement with ICC values of 0.93 [95%CI = 0.92-0.94] for AI-I and 0.89 [95%CI = 0.86-0.91] for AI-II. AI-II differed from AI-III in malignancy prediction with AUC = 0.77 [95%CI = 0.72-0.81] and 0.89 [95%CI = 0.85-0.92], respectively. Additionally, AI-II showed significantly lower PPV (65.57% vs. 87.20%, <i>p</i> < 0.001) and accuracy (72.1% vs. 82%, <i>p</i> < 0.001) than AI-III. AI-based tools demonstrated high accuracy for incidental pulmonary detection; however, their performance in malignancy risk stratification differed substantially.