A semi-automated quality assurance tool for cardiovascular magnetic resonance imaging: application to outlier detection, artificial intelligence evaluation and trainee feedback.
Authors
Affiliations (12)
Affiliations (12)
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, ECRC Experimental and Clinical Research Center, Berlin, Germany. [email protected].
- Working Group on Cardiovascular Magnetic Resonance, Experimental and Clinical Research Center, A Joint Cooperation Between the Charité - Universitätsmedizin Berlin and the Max-Delbrück-Center for Molecular Medicine, Berlin, Germany. [email protected].
- DZHK (German Centre for Cardiovascular Research), Partner Site Berlin, Berlin, Germany. [email protected].
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, ECRC Experimental and Clinical Research Center, Berlin, Germany.
- Working Group on Cardiovascular Magnetic Resonance, Experimental and Clinical Research Center, A Joint Cooperation Between the Charité - Universitätsmedizin Berlin and the Max-Delbrück-Center for Molecular Medicine, Berlin, Germany.
- DZHK (German Centre for Cardiovascular Research), Partner Site Berlin, Berlin, Germany.
- Deutsches Herzzentrum der Charité-Medical Heart Center of Charité and German Heart Institute Berlin, Klinik für Kardiologie, Angiologie und Intensivmedizin, Berlin, Germany.
- Department of Anesthesiology and Operative Intensive Care Medicine, Charité Campus Benjamin Franklin, Hindenburgdamm 30, 12203, Berlin, Germany.
- Deutsches Herzzentrum der Charité (DHZC), Institute of Computer-Assisted Cardiovascular Medicine, Augustenburger Platz 1, Berlin, Germany.
- Fraunhofer MEVIS, Bremen, Germany.
- Department of Computer Sciences, Hochschule Darmstadt - University of Applied Sciences, Darmstadt, Germany.
- Department of Cardiology and Nephrology, Helios Hospital Berlin-Buch, Berlin, Germany.
Abstract
Cardiovascular magnetic resonance (CMR) offers state-of-the-art volume, function, fibrosis and oedema imaging. Quality assurance (QA) tasks, such as quantitative parameter reproducibility assessments, the evaluation of AI methods, and the assessment of trainees have become essential to CMR. However, the explainability of how qualitative differences impact quantitative differences remains underexplored. Our aim is to demonstrate a semi-automated QA tool, Lazy Luna's (LL) applicability to typical CMR QA application cases. A software feature error-tracing is designed that allows for quickly pinpointing qualitative reasons for quantitative differences and outliers. Three QA application cases were designed. First, LL was applied to perform outlier detection for inter- and intraobserver analyses to detect failure cases and provide qualitative explanations. Outlier detection was performed on several typical images types. Second, LL supported an Artificial intelligence (AI) evaluation, in which an AI method was compared to a CMR-expert of 144 patients. LL assessed the acceptability of AI biases for left and right ventricular (LV, RV) end-systolic, -diastolic, and stroke volumes (ESV, EDV, SV), ejection fractions (EF) and the myocardial mass (LVM). Annotations were examined to explain the qualitative differences that resulted in good and poor parameters. The AI investigation was recorded as a video. Third, LL was used to provide a Trainee Feedback to a CMR beginner. The trainee was compared to an expert on several imaging techniques to investigate outliers. For the outlier detection, LL detected segmentation differences that caused parameter differences on multiple sequences. For the AI evaluation calculated clinical parameter biases to be: LVESV:-3.1 ml, LVEDV:2.1 ml, LVSV:6.5 ml, LVEF:3.0 ml, RVESV:0.3 ml, RVEDV:-3.8 ml, RVSV:-4.2 ml, RVEF:-1.4 ml, LVM:-2 g. Inspecting the causes for outlier differences revealed that juxtaposed basal slice failures caused unacceptable LVSV deviations between AI and expert. For the trainee assessment, LL showed that trainee parameters exceeded tolerance ranges. The segmentations could be improved to better mirror expert segmentations and close the parameter gaps. Lazy Luna, as a semi-automated quality assurance tool, is applicable to several quality assurance application cases in CMR.