Examining explainable artificial intelligence in TNM staging with PET-CT: a user-centred observation study.
Authors
Affiliations (13)
Affiliations (13)
- Centre for Vision, Speech and Signal Processing, The University of Surrey, Guildford, UK. [email protected].
- Mirada Medical Ltd., Oxford, UK. [email protected].
- Mirada Medical Ltd., Oxford, UK.
- Centre for Vision, Speech and Signal Processing, The University of Surrey, Guildford, UK.
- Royal Surrey NHS Foundation Trust, Guildford, UK.
- Alliance Medical Ltd., Warwick, UK.
- The Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, UK.
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
- East Kent Hospitals University NHS Foundation Trust, Canterbury, UK.
- Leeds Teaching Hospitals NHS Trust, Leeds, UK.
- Faculty of Medicine and Health, University of Leeds, Leeds, UK.
- King's College London NHS Foundation Trust, London, UK.
- The Royal Wolverhampton NHS Trust, Wolverhampton, UK.
Abstract
Artificial intelligence (AI) is increasingly proposed as a solution to improve efficiency in radiology and nuclear medicine, particularly in the context of workforce shortages. However, adoption of AI-based clinical decision support systems (AI-CDSS) remains slow, due to limited model transparency. Explainable AI (XAI) may improve clinician acceptance by supporting oversight and trust. This study evaluated the impact of different XAI explanation types on radiologists' willingness to adopt AI systems. Ten nuclear medicine radiologists from eight UK institutions performed lung cancer TNM staging using whole-body PET/CT scans supported by a simulated AI-CDSS. Three explanation approaches were assessed: input feature attribution, high-level concept explanations and global algorithmic transparency. Adoption likelihood and explanation usefulness were rated using Likert scales and analysed with nonparametric sign tests. Semi-structured interviews were additionally analysed through thematic evaluation supported by large language model-assisted coding with human verification. All explanation approaches significantly increased radiologists' willingness to adopt the AI system compared to a black-box model (p < 0.05). Explanations were consistently considered useful in enabling participants to confirm or challenge AI staging recommendations (p < 0.001). Qualitative findings highlighted the importance of clinical relevance, error detection and decision support value. A trade-off between explanation depth and usability was identified as a key factor influencing preferences. Incorporating XAI into nuclear medicine CDSS enhances radiologists' acceptance and provides clinically meaningful information for oversight of AI recommendations, in accordance with the EU AI Act. These findings support the role of XAI in facilitating integration of AI tools into diagnostic workflows.