Artificial Intelligence Language Models to Translate Professional Radiology Mammography Reports Into Plain Language - Impact on Interpretability and Perception by Patients.
Authors
Affiliations (3)
Affiliations (3)
- Department of Radiology, Kantonsspital Baden, affiliated Hospital for Research and Teaching of the Faculty of Medicine of the University of Zurich, Baden, Switzerland (D.P., M.K., J.H., M.F., R.A.K.H., A.E.).
- Department of Gynecology, Interdisciplinary Breast Center, Kantonsspital Baden, affiliated Hospital for Research and Teaching of the Faculty of Medicine of the University of Zurich, Baden, Switzerland (C.L.).
- Department of Radiology, Kantonsspital Baden, affiliated Hospital for Research and Teaching of the Faculty of Medicine of the University of Zurich, Baden, Switzerland (D.P., M.K., J.H., M.F., R.A.K.H., A.E.). Electronic address: [email protected].
Abstract
This study aimed to evaluate the interpretability and patient perception of AI-translated mammography and sonography reports, focusing on comprehensibility, follow-up recommendations, and conveyed empathy using a survey. In this observational study, three fictional mammography and sonography reports with BI-RADS categories 3, 4, and 5 were created. These reports were repeatedly translated to plain language by three different large language models (LLM: ChatGPT-4, ChatGPT-4o, Google Gemini). In a first step, the best of these repeatedly translated reports for each BI-RADS category and LLM was selected by two experts in breast imaging considering factual correctness, completeness, and quality. In a second step, female participants compared and rated the translated reports regarding comprehensibility, follow-up recommendations, conveyed empathy, and additional value of each report using a survey with Likert scales. Statistical analysis included cumulative link mixed models and the Plackett-Luce model for ranking preferences. 40 females participated in the survey. GPT-4 and GPT-4o were rated significantly higher than Gemini across all categories (P<.001). Participants >50 years of age rated the reports significantly higher as compared to participants of 18-29 years of age (P<.05). Higher education predicted lower ratings (P=.02). No prior mammography increased scores (P=.03), and AI-experience had no effect (P=.88). Ranking analysis showed GPT-4o as the most preferred (P=.48), followed by GPT-4 (P=.37), with Gemini ranked last (P=.15). Patient preference differed among AI-translated radiology reports. Compared to a traditional report using radiological language, AI-translated reports add value for patients, enhance comprehensibility and empathy and therefore hold the potential to improve patient communication in breast imaging.