The Evolution of Radiology Image Annotation in the Era of Large Language Models.
Authors
Affiliations (6)
Affiliations (6)
- Department of Radiology, Thomas Jefferson University, 132 S Tenth St, Ste 1080 B Main Building, Philadelphia, PA 19107.
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY.
- Department of Thoracic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex.
- Department of Diagnostic Imaging, Universidade Federal de São Paulo, São Paulo, Brazil.
- Department of Radiology, Weill Cornell Medical College, New York, NY.
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, Calif.
Abstract
Although there are relatively few diverse, high-quality medical imaging datasets on which to train computer vision artificial intelligence models, even fewer datasets contain expertly classified observations that can be repurposed to train or test such models. The traditional annotation process is laborious and time-consuming. Repurposing annotations and consolidating similar types of annotations from disparate sources has never been practical. Until recently, the use of natural language processing to convert a clinical radiology report into labels required custom training of a language model for each use case. Newer technologies such as large language models have made it possible to generate accurate and normalized labels at scale, using only clinical reports and specific prompt engineering. The combination of automatically generated labels extracted and normalized from reports in conjunction with foundational image models provides a means to create labels for model training. This article provides a short history and review of the annotation and labeling process of medical images, from the traditional manual methods to the newest semiautomated methods that provide a more scalable solution for creating useful models more efficiently. <b>Keywords:</b> Feature Detection, Diagnosis, Semi-supervised Learning © RSNA, 2025.