Design for Understanding: How Format Influences Comprehension and Confidence in AI-Generated Patient-Friendly Radiology Reports.
Authors
Affiliations (3)
Affiliations (3)
- Department of Radiology, Stanford University, Palo Alto, California, United States; AI Development and Evaluation Lab, Stanford University, Palo Alto, California, United States. Electronic address: [email protected].
- Department of Radiology, Stanford University, Palo Alto, California, United States; AI Development and Evaluation Lab, Stanford University, Palo Alto, California, United States.
- Department of Radiology, Stanford University, Palo Alto, California, United States.
Abstract
Determine whether the format of AI-generated patient-friendly reports (PFRs) affects laypeople's comprehension of medical information. In a within-subjects study, 116 participants viewed radiology reports alongside AI-generated PFR versions in three formats: dictionary (term definitions), block (full translation), and a novel sentence-based format (paired source-translation sentences for cross-referencing). Objective and subjective comprehension, confidence, and response time were measured. Relative to the dictionary format, objective comprehension odds were higher for sentence (OR 2.31, [1.36-3.94]) and block (OR 2.23, [1.30-3.80]), and subjective comprehension was higher for sentence (OR 4.63, [2.24-9.57]) and block (OR 2.18, [1.13-4.23]). Response confidence was higher for sentence (OR 3.75, [2.41-5.84]) and block (OR 2.21, [1.45-3.35]) than dictionary, with sentence higher than block (p_adj<0.05). Confidence decreased on later trials (OR 0.82, [0.69-0.97]) and for longer reports (OR 0.55, [0.41-0.75]). Response times were faster for sentence (ratio 0.74, [0.66-0.83]) and block (ratio 0.71, [0.64-0.80]) than dictionary; longer reports and higher health-confidence slowed responses. Higher response confidence associated with greater objective (OR 2.75, [1.84-4.13]; p<0.001) and subjective comprehension (b=0.59; p<0.001). Presentation format is a key, modifiable factor influencing how laypeople interpret AI-generated PFRs. Sentence and block formats showed higher comprehension, confidence, and faster responses than a dictionary format. The novel sentence-based format elicited the highest confidence, suggesting that cross-referencing between AI output and source text fosters trust and supports understanding. These findings highlight the critical role of user interface design in effective AI-generated patient communication.