Optimization of deep learning models for inference in low resource environments.
Authors
Affiliations (13)
Affiliations (13)
- Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA. Electronic address: [email protected].
- Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA; Department of Informatics, Technical University of Munich, Munich, Germany. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Intel Corporation, Santa Clara, CA, USA. Electronic address: [email protected].
- Department of Computer Science, School of Computer Science & Mathematics (CSM), Kingston University, London, UK. Electronic address: [email protected].
- Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA; Department of Computer Science, School of Computer Science & Mathematics (CSM), Kingston University, London, UK; Department of Radiology & Imaging Sciences, Indiana University School of Medicine, Indianapolis, IN, USA; Department of Biostatistics & Health Data Science, Indiana University School of Medicine, Indianapolis, IN, USA; Department of Neurological Surgery, Indiana University School of Medicine, Indianapolis, IN, USA. Electronic address: [email protected].
Abstract
Artificial Intelligence (AI), and particularly deep learning (DL), has shown great promise to revolutionize healthcare. However, clinical translation is often hindered by demanding hardware requirements. In this study, we assess the effectiveness of optimization techniques for DL models in healthcare applications, targeting varying AI workloads across the domains of radiology, histopathology, and medical RGB imaging, while evaluating across hardware configurations. The assessed AI workloads focus on both segmentation and classification workloads, by virtue of brain extraction in Magnetic Resonance Imaging (MRI), colorectal cancer delineation in Hematoxylin & Eosin (H&E) stained digitized tissue sections, and diabetic foot ulcer classification in RGB images. We quantitatively evaluate model performance in terms of model runtime during inference (including speedup, latency, and memory usage) and model utility on unseen data. Our results demonstrate that optimization techniques can substantially improve model runtime, without compromising model utility. These findings suggest that optimization techniques can facilitate the clinical translation of AI models in low-resource environments, making them more practical for real-world healthcare applications even in underserved regions.