Evaluation of the impact of artificial intelligence-assisted image interpretation on the diagnostic performance of clinicians in identifying endotracheal tube position on plain chest X-ray: a multi-case multi-reader study.
Authors
Affiliations (16)
Affiliations (16)
- Oxford Clinical Artificial Intelligence Research (OxCAIR), Oxford University Hospitals NHS Foundation Trust, Oxford, UK. [email protected].
- Oxford Clinical Artificial Intelligence Research (OxCAIR), Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
- Royal Cornwall Hospitals NHS Trust, Cornwall, UK.
- Department of Imaging, Queen Elizabeth University Hospital, Glasgow, UK.
- NHS Greater Glasgow and Clyde, Glasgow, UK.
- Department of Anaesthesia, Hammersmith Hospital, Imperial College Healthcare NHS Trust, London, UK.
- Department of Anaesthesia, University College London Hospitals NHS Foundation Trust, London, UK.
- Department of Targeted Intervention, University College London, London, UK.
- GE Healthcare (GEHC), Chalfont St. Giles, UK.
- The University of Oxford, Oxford, UK.
- Reporting and Image Quality Control Ltd, Oxford, UK.
- Emergency Medicine Research Oxford (EMROx), Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
- Buckinghamshire Healthcare NHS Trust, Aylesbury, UK.
- NHS Frimley Health Foundation Trust, Frimley, UK.
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
- Royal Berkshire NHS Foundation Trust, Reading, UK.
Abstract
Incorrectly placed endotracheal tubes (ETTs) can lead to serious clinical harm. Studies have demonstrated the potential for artificial intelligence (AI)-led algorithms to detect ETT placement on chest X-Ray (CXR) images, however their effect on clinician accuracy remains unexplored. This study measured the impact of an AI-assisted ETT detection algorithm on the ability of clinical staff to correctly identify ETT misplacement on CXR images. Four hundred CXRs of intubated adult patients were retrospectively sourced from the John Radcliffe Hospital (Oxford) and two other UK NHS hospitals. Images were de-identified and selected from a range of clinical settings, including the intensive care unit (ICU) and emergency department (ED). Each image was independently reported by a panel of thoracic radiologists, whose consensus classification of ETT placement (correct, too low [distal], or too high [proximal]) served as the reference standard for the study. Correct ETT position was defined as the tip located 3-7 cm above the carina, in line with established guidelines. Eighteen clinical readers of varying seniority from six clinical specialties were recruited across four NHS hospitals. Readers viewed the dataset using an online platform and recorded a blinded classification of ETT position for each image. After a four-week washout period, this was repeated with assistance from an AI-assisted image interpretation tool. Reader accuracy, reported confidence, and timings were measured during each study phase. 14,400 image interpretations were undertaken. Pooled accuracy for tube placement classification improved from 73.6 to 77.4% (p = 0.002). Accuracy for identification of critically misplaced tubes increased from 79.3 to 89.0% (p = 0.001). Reader confidence improved with AI assistance, with no change in mean interpretation time at 36 s per image. Use of assistive AI technology improved accuracy and confidence in interpreting ETT placement on CXR, especially for identification of critically misplaced tubes. AI assistance may potentially provide a useful adjunct to support clinicians in identifying misplaced ETTs on CXR.