Enhanced bi-branch deep learning network for in vivo hyperspectral imaging recognition of organs and tissues.
Authors
Affiliations (3)
Affiliations (3)
- Research Center for Analytical Sciences, Tianjin Key Laboratory of Biosensing and Molecular Recognition, State Key Laboratory of Medicinal Chemical Biology, College of Chemistry, Nankai University, Tianjin 300071, China.
- Research Center for Analytical Sciences, Tianjin Key Laboratory of Biosensing and Molecular Recognition, State Key Laboratory of Medicinal Chemical Biology, College of Chemistry, Nankai University, Tianjin 300071, China; Haihe Laboratory of Sustainable Chemical Transformations, Tianjin 300192, China.
- Research Center for Analytical Sciences, Tianjin Key Laboratory of Biosensing and Molecular Recognition, State Key Laboratory of Medicinal Chemical Biology, College of Chemistry, Nankai University, Tianjin 300071, China; Haihe Laboratory of Sustainable Chemical Transformations, Tianjin 300192, China. Electronic address: [email protected].
Abstract
Hyperspectral imaging, as an emerging medical imaging technology, offers significant potential in biomedical research due to its ability in capturing rich spectral information. An enhanced bi-branch network integrating graph convolutional network (GCN) and transformer for hyperspectral image recognition of multiple organs and tissues was developed. GCN is able to extract regional information of same category adjacent to the target pixel, while transformer utilizes long-range dependencies to capture the boundary information. Furthermore, feature enhancement modules were incorporated to improve the performance of the model, and the efficiency was shown by a comparison of the feature distributions obtained with and without these modules. Finally, the model was validated using a public porcine dataset and was applied to in vivo organ and tissue identification from the hyperspectral images of zebrafish measured by diffuse reflectance near-infrared spectroscopy. The proposed model was found able to achieve a higher precision in recognizing the organ and tissue structures with clear and continuous boundaries.