A knowledge-guided and uncertainty-calibrated multimodal framework for fracture diagnosis and radiology report generation.

Authors

Abstract

Fracture diagnosis from radiographic imaging remains challenging, particularly in clinical settings with limited access to expert radiologists or standardized reporting practices. This work introduces UG-GraphT5 (Uncertainty-Guided Graph Transformer for Radiology Report Generation), a unified multimodal framework for joint fracture classification and uncertainty-aware radiology report generation that explicitly treats diagnostic uncertainty as a central component guiding both reasoning and clinical communication. The proposed approach integrates visual representations, structured clinical knowledge derived from SNOMED CT, Bayesian uncertainty estimation, and guided natural language generation based on ClinicalT5, enabling adaptive multimodal fusion and calibrated language output. Evaluated on three radiological datasets comprising over 80,000 expert-annotated images and reports, UG-GraphT5 achieves improved fracture classification performance (F1-score of 82.6%), strong uncertainty calibration (ECE of 2.7%), and high-quality report generation (BLEU-4 of 0.356). Qualitative analysis and a reader study involving radiology trainees and experts further confirm that generated reports appropriately reflect diagnostic confidence through uncertainty-aware lexical modulation. An optimized clinical inference profile reduces inference latency by more than 40% without compromising diagnostic accuracy, highlighting the framework's potential for interpretable, trustworthy, and deployment-aware AI-assisted radiology in resource-constrained clinical environments.

Tags

Topics