Large language model based simplification of ophthalmological clinical and ancillary results for patient readability.
Authors
Affiliations (5)
Affiliations (5)
- University of São Paulo (USP). MBA, Pontifical Catholic University of Rio Grande do Sul (PUC-RS), Federal University of Pará (UFPA), São Paulo, Brazil.
- Federal University of Pará (UFPA), Belém, Brazil.
- Division of Ophthalmology, Faculdade de Medicina FMUSP, University of São Paulo (USP), Av. Dr Enéas de Carvalho Aguiar, 255, Cerqueira César, 05403-001, São Paulo, Brazil.
- Retina Prime, Aracaju, Brazil.
- Division of Ophthalmology, Faculdade de Medicina FMUSP, University of São Paulo (USP), Av. Dr Enéas de Carvalho Aguiar, 255, Cerqueira César, 05403-001, São Paulo, Brazil. [email protected].
Abstract
Ophthalmological reports are often written at a complexity level that exceeds the reading ability of many patients. Large language models (LLMs) may help simplify these texts, but their performance depends on prompt design and must preserve clinical fidelity. This study evaluated whether different prompting strategies improve the readability and safety of simplified ophthalmological reports. We analyzed 443 de-identified reports from a tele-ophthalmology platform, including 280 retinal fundus and 163 ocular ultrasound reports. Each report was simplified by four LLMs (GPT-3.5, GPT-4.0, Gemini, and Copilot) using three prompts: a basic command, a patient-oriented prompt, and a targeted 7th-grade reading level prompt. Readability was measured with the average Readability Grade Level (aRGL). A representative eligible subset of the generated texts was independently evaluated by two third-year ophthalmology residents, and each selected response underwent independent review by both evaluators, with disagreements resolved by a board-certified retina ophthalmologist. Outcomes included factual accuracy, information completeness, and potential for harm. The original reports were highly complex, with a median aRGL of 13.8 overall, 11.6 for fundus reports, and 15.4 for ocular ultrasound reports. All LLMs improved readability scores to varying degrees. Prompt engineering was a major determinant of performance, and the targeted 7th-grade prompt produced the best results across models. Copilot, Gemini, and GPT-4.0 achieved median aRGL values closest to the recommended patient-facing reading level, while GPT-3.5 showed weaker performance in some comparisons. Clinical validation showed substantial agreement between raters (kappa range, 0.70-0.97). After adjudication, 91% of simplified texts were factually accurate, 86% retained all critical information, and 95% showed no potential for harm. LLMs can simplify ophthalmological reports while preserving clinical fidelity, but performance depends strongly on prompt specificity. These tools show promise for patient-facing communication, although human oversight remains essential.