Implementation of an automated contour quality assurance tool within the TROG 18.01 NINJA trial.
Authors
Affiliations (11)
Affiliations (11)
- South West Sydney Clinical Campuses, University of New South Wales, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia; Liverpool and Macarthur Cancer Therapy Centre, Liverpool, NSW, Australia. Electronic address: [email protected].
- South West Sydney Clinical Campuses, University of New South Wales, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia; Australian e-Health Research Centre- CSIRO, Brisbane, QLD, Australia.
- Calvary Mater Newcastle, Newcastle, NSW, Australia; School of Medicine and Public Health, University of Newcastle, Newcastle, NSW, Australia.
- South West Sydney Clinical Campuses, University of New South Wales, Sydney, NSW, Australia; Liverpool and Macarthur Cancer Therapy Centre, Liverpool, NSW, Australia.
- Liverpool and Macarthur Cancer Therapy Centre, Liverpool, NSW, Australia.
- Trans-Tasman Radiation Oncology Group, Newcastle, NSW, Australia.
- South West Sydney Clinical Campuses, University of New South Wales, Sydney, NSW, Australia; Australian e-Health Research Centre- CSIRO, Brisbane, QLD, Australia.
- South West Sydney Clinical Campuses, University of New South Wales, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia; Liverpool and Macarthur Cancer Therapy Centre, Liverpool, NSW, Australia.
- Institute for Medical Physics, School of Physics, The University of Sydney, Australia.
- Sir Charles Gairdner Hospital, Perth, WA, Australia; University of Western Australia, Perth, WA, Australia; University of Wisconsin, Madison, WI, USA.
- South West Sydney Clinical Campuses, University of New South Wales, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia; Liverpool and Macarthur Cancer Therapy Centre, Liverpool, NSW, Australia; Institute for Medical Physics, School of Physics, The University of Sydney, Australia.
Abstract
Automated contour quality assurance (QA) has the potential to reduce resource and cost requirements for clinical trial QA. While several studies have developed such models, few have been translated for use in prospective real-world trials. This study aimed to implement a contour QA tool using a previously developed model and evaluate its performance during deployment within the TROG 18.01 NINJA prostate cancer trial. A software tool was developed using an existing prostate clinical target volume (CTV) QA model and integrated into the trial QA workflow. A pilot study was conducted over an 18-month period, during which 56 CTVs were assessed. Reports generated by the tool flagged cases for review and were provided to radiation oncologists to support QA processes. All five protocol-violating CTVs were correctly identified during deployment, yielding a sensitivity of 1.0. However, a higher-than-expected false positive rate resulted in an accuracy of 0.46 and specificity of 0.41. Retrospective analysis showed that many cases submitted deviated from the model's training distribution, primarily due to inconsistencies in MRI acquisition and variation in submitted CTV definitions. Incorporating out-of-distribution detection based on histogram correlation and model uncertainty improved accuracy to 0.69 for in-distribution cases. Radiation oncologists reported time savings of up to 60 min per case. However, preparation of data remained time intensive for QA coordinators, highlighting the need for further workflow automation. These findings support the feasibility of automated contour QA in multicentre trials and offer guidance for future implementation at scale.