CLEAR: an auditable foundation model for radiology grounded in clinical concepts.
Authors
Affiliations (14)
Affiliations (14)
- Center for AI and Data Science for Integrated Diagnostics (AI2D), Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. [email protected].
- Department of Diagnostic Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. [email protected].
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. [email protected].
- Artificial Intelligence in Biomedical Imaging Laboratory (AIBIL), Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany.
- Department of Diagnostic and Interventional Radiology, School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany.
- Department of Cardiovascular Radiology and Nuclear Medicine, School of Medicine and Health, German Heart Center, Technical University of Munich, Munich, Germany.
- Center for AI and Data Science for Integrated Diagnostics (AI2D), Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany.
- National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany.
- Department of Medical Oncology, Heidelberg University Hospital, Heidelberg, Germany.
- Department of Medicine I, University Hospital Dresden, Dresden, Germany.
- Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Diagnostic Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
Abstract
'Black box' deep learning models for medical image interpretation limit clinical trust and analysis of performance degradation. Here we introduce Concept-Level Embeddings for Auditable Radiology (CLEAR), an auditable foundation model based on clinical concepts. Trained on over 0.87 million image-report pairs from 239,391 patients, CLEAR learns a visual representation and projects chest X-rays into a semantically rich space defined by large language model embeddings, making every prediction decomposable into weighted contributions from individual radiological observations. External validation on four large, physician-annotated datasets from the United States, Europe and Asia shows that CLEAR not only achieves state-of-the-art classification performance but also enables applications: auditable zero-shot pathology detection, systematic identification of radiological confounders and the creation of expert-level concept bottleneck models from data-driven concepts. By integrating clinical knowledge directly into its reasoning process, CLEAR offers a framework for robust model auditing, safer deployment and enhanced physician-AI collaboration, advancing towards trustworthy medical AI.