Tongue Coating Grading Identification Using Deep Learning for Hyperspectral Imaging Data

Tongue diagnosis is one of the four diagnostic methods of traditional Chinese medicine (TCM), which has important value in clinical disease diagnosis and efficacy evaluation. The change in tongue coating is a comprehensive cause of multi-dimensional changes such as color, texture, and substance. However, the color tongue image contains less spectral information, which may lead to the lack of key information in tongue diagnosis. Hyperspectral images can obtain reflection information of tongue images in hundreds of spectral bands. Unlike traditional color images, the rich spectral information can more accurately and sensitively describe and classify tongue coating, and has been widely applied in biomedical images. In this paper, we conducted feature extraction and analysis on hyperspectral images of different tongue coatings, and proposed a spectral-spatial feature deep learning framework to classification and quantitative recognition the tongue coating based on hyperspectral image features. Firstly, 360 hyperspectral images of tongue body were collected, and clinicians were identify all tongue coatings and divided them into 6 different grades. The hyperspectral features of each tongue coating area were extracted respectively. In order to reduce noise interference, singular spectrum analysis was used to preprocess the hyperspectral curve features. Considering the actual situation of tongue coating, a depth learning model was established to analyze the spectral and spatial feature of the hyperspectral tongue image to identified the grading of tongue coating. The experimental results showed that tongue coating with different quantization levels had different hyperspectral features, and the recognition rate of tongue coating quantization level can reach 87.21% using the spatial-spectral features.