A multi-centre real-world evaluation of AI-assisted organ at risk contouring on radiotherapy treatment planning workflows.
Authors
Affiliations (12)
Affiliations (12)
- King's Technology Evaluation Centre, London Institute for Healthcare Engineering, King's College London, SE1 7AR, UK.
- Medical Physics and Clinical Engineering, Medical Physics, Guy's and St Thomas' NHS Foundation Trust, SE1 9RT, UK. King's Technology Evaluation Centre, London Institute for Healthcare Engineering, King's College London, SE1 7AR, UK.
- Centre for Rheumatic Diseases, Department of Inflammation Biology, School of Immunology and Microbial Sciences, Faculty of Life Sciences and Medicine, King's College London, SE5 9RJ, UK.
- Cancer National Programme of Care, Specialised Commissioning, NHS England, UK.
- Specialised Commissioning Team, NHS England, UK.
- Inpictura Ltd, Abingdon OX14 3PX, UK. Division of Cancer Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, M20 4BX, UK.
- Radiotherapy Physics, Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, NE7 7DN, UK.
- Radiotherapy Physics, Nottingham University Hospitals NHS Trust, NG5 1PB, UK.
- Research Economics and Consultancy in Healthcare Ltd., Solihull, UK.
- School of Life Course and Population Sciences, King's College London, London, SE1 1UL, UK.
- Department of Public Services Management & Organisation, King's Business School, King's College London, WC2B 4BG, UK.
- Clinical Engineer, Medical Physics, Guy's and St Thomas' NHS Foundation Trust, SE1 9RT, UK. King's Technology Evaluation Centre, London Institute for Healthcare Engineering, King's College London, SE1 7AR, UK.
Abstract
This multicentre real-world evaluation, commissioned by NHS England, evaluated the impact of AI-assisted contouring of organs at risk on contour acceptability, workflows and staffing. Data from 626 patients were collected from eight NHS radiotherapy departments. Time metrics from date of planning CT scan to start of first treatment were compared between manual and AI-enabled pathways. Acceptability scores for AI-generated contours were also collected. AI-assisted contouring increased potential efficiency in treatment planning compared to manual methods, reducing time for contouring and redistributing workload across different staff types. 100% manual contour reviews involved clinical oncologists compared to 38% reviewing AI-generated contours. However, workflow design meant that time saving was not observed across the whole pathway to start of treatment.AI contours were generally well accepted, with 15.9% requiring no edits and 64.4% only minor edits. This varied by anatomy, with breast having the best acceptability and prostate and head and neck contours requiring more editing. AI contouring tools have the potential to enhance efficiency in radiotherapy treatment planning, creating operational flexibility. Pathway review and revision could unlock further benefits, for example, involving different staff types to address local bottlenecks. Workflow, capacity and staffing review pre- and post-implementation could increase efficiency gains with AI-assisted contouring. This study evaluated AI contouring tools in complex, real-world systems which differ between departments for generalisable conclusions. It describes the changes in time across the whole treatment planning pathway and system-level impact of using AI tools to help inform holistic planning.