Image-Based Search in Radiology: Identification of Brain Tumor Subtypes within Databases Using MRI-Based Radiomic Features.
Authors
Affiliations (14)
Affiliations (14)
- From the University of Leipzig (M.v.R., K.W., K.-T.H., S.A.), Leipzig, Germany.
- Department of Radiology and Biomedical Imaging (S.C., A.A.), Yale School of Medicine, New Haven, Connecticut.
- Center for Outcomes Research and Evaluation (S.C.), Yale School of Medicine, New Haven, Connecticut.
- Department of Radiology (A.A.), Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts.
- Department of Radiology (N.M., M.S.A.), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.
- Center for Translational Imaging Analysis and Machine Learning, Department of Radiology and Biomedical Imaging (T.Z.), Yale School of Medicine, New Haven, Connecticut.
- Department of Neurosurgery (J.L.), Heinrich-Heine-University, Duesseldorf, Germany.
- Department of Diagnostic and Interventional Radiology, Medical Faculty (N.T.), University Dusseldorf, Dusseldorf, Germany.
- University of Duisburg-Essen (L.J.), Essen, Germany.
- DKFZ Division of Translational Neuro-oncology at the WTZ, German Cancer Consortium, DKTK Partner Site (L.J.), University Hospital Essen, Essen, Germany.
- University of Ulm (S.M.), Ulm, Germany.
- Visage Imaging, Inc. (M.L.), San Diego, California.
- Department of Therapeutic Radiology (S.A.), Yale School of Medicine, New Haven, Connecticut.
- Department of Radiology (N.M., M.S.A.), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania [email protected].
Abstract
Existing neuroradiology reference materials do not cover the full range of primary brain tumor presentations, and text-based medical image search engines are limited by the lack of consistent structure in radiology reports. To address this, an image-based search approach is introduced here, leveraging an institutional database to find reference MRIs visually similar to presented query cases. Two hundred ninety-five patients (mean age and standard deviation, 51 ± 20 years) with primary brain tumors who underwent surgical and/or radiotherapeutic treatment between 2000 and 2021 were included in this retrospective study. Semiautomated convolutional neural network-based tumor segmentation was performed, and radiomic features were extracted. The data set was split into reference and query subsets, and dimensionality reduction was applied to cluster reference cases. Radiomic features extracted from each query case were projected onto the clustered reference cases, and nearest neighbors were retrieved. Retrieval performance was evaluated by using mean average precision at k, and the best-performing dimensionality reduction technique was identified. Expert readers independently rated visual similarity by using a 5-point Likert scale. t-Distributed stochastic neighbor embedding with 6 components was the highest-performing dimensionality reduction technique, with mean average precision at 5 ranging from 78%-100% by tumor type. The top 5 retrieved reference cases showed high visual similarity Likert scores with corresponding query cases (76% 'similar' or 'very similar'). We introduce an image-based search method for exploring historical MR images of primary brain tumors and fetching reference cases closely resembling queried ones. Assessment involving comparison of tumor types and visual similarity Likert scoring by expert neuroradiologists validates the effectiveness of this method.