Best Practices and Checklist for Reviewing Artificial Intelligence-Based Medical Imaging Papers: Classification.
Authors
Affiliations (21)
Affiliations (21)
- Department of Radiology, Mayo Clinic, Rochester, MN, USA. [email protected].
- Bunkerhill Health, Palo Alto, CA, USA.
- Universidade Federal de São Paulo, São Paulo, Brazil.
- Carle College of Medicine University of Illinois Urbana-Champaign, Urbana, IL, USA.
- Imaging Center Onex, Groupe 3R, Geneva, Switzerland.
- Singularity Consulting, Geneva, Switzerland.
- Microsoft Research, Cambridge, MA, USA.
- Department of Radiology, NYU School of Medicine, New York, NY, USA.
- Department of Radiology, Emory University School of Medicine, Atlanta, GA, USA.
- Ionic Health, Porto, Portugal.
- Johns Hopkins University, Baltimore, MD, USA.
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
- Children's Hospital of Philadelphia, Philadelphia, PA, USA.
- UPMC Children's Hospital, Pittsburgh, PA, USA.
- De La Salle University, Manila, Luzon, Philippines.
- Hospital Selayang, Batu Caves, Selangor, Malaysia.
- GE Healthcare, JFWTC, Bangalore, Karnataka, India.
- Department of Radiology, Neuroradiology Division Memorial Sloan Kettering Cancer Center, New York, NY, USA.
- , Aidoc, Tel Aviv, Israel.
- Department of Radiology, Weill Medical College of Cornell University, New York, NY, USA.
- Department of Radiology, Mayo Clinic, Rochester, MN, USA.
Abstract
Recent advances in Artificial Intelligence (AI) methodologies and their application to medical imaging has led to an explosion of related research programs utilizing AI to produce state-of-the-art classification performance. Ideally, research culminates in dissemination of the findings in peer-reviewed journals. To date, acceptance or rejection criteria are often subjective; however, reproducible science requires reproducible review. The Machine Learning Education Sub-Committee of the Society for Imaging Informatics in Medicine (SIIM) has identified a knowledge gap and need to establish guidelines for reviewing these studies. This present work, written from the machine learning practitioner standpoint, follows a similar approach to our previous paper related to segmentation. In this series, the committee will address best practices to follow in AI-based studies and present the required sections with examples and discussion of requirements to make the studies cohesive, reproducible, accurate, and self-contained. This entry in the series focuses on image classification. Elements like dataset curation, data pre-processing steps, reference standard identification, data partitioning, model architecture, and training are discussed. Sections are presented as in a typical manuscript. The content describes the information necessary to ensure the study is of sufficient quality for publication consideration and, compared with other checklists, provides a focused approach with application to image classification tasks. The goal of this series is to provide resources to not only help improve the review process for AI-based medical imaging papers, but to facilitate a standard for the information that should be presented within all components of the research study.