SSL-based Chewing and Swallowing Detection Using Multiple Skin-contact Microphones

The recognition of eating behavior has a crucial role to play in the health management and monitoring of the elderly. Here, we propose an approach to the automatic detection of chewing and swallowing using skin-contact microphones. To improve the accuracy of the detection of chewing and swallowing, we explored the use of both self-supervised learning (SSL) and integrating information from multiple skin-contact microphones at different locations (2 channels below the ears and 2 channels on the throat). We evaluated our approach with training data collected from 27 subjects and test data from 5 subjects who did not contribute to the training data. We achieved a high-accuracy, with a character error rate of 2.88%, using an SSL-based model with a 4-channel summed signal. Furthermore, a frame-level F1-score evaluation showed a high degree of detection accuracy for both chewing and swallowing, achieving F1-scores of 0.954 for chewing and 0.97 for swallowing, with a 0.1 s allowance. These results demonstrate that SSL-based speech recognition models can be effectively applied to the detection of chewing and swallowing, even using a simple Early Fusion approach.