Enhancing Intake Monitoring: Transfer Learning for Audio-Based Detection of Swallowing Events

Advancements in sensing technology and deep learning have paved the way for non-invasive monitoring solutions in healthcare. This paper reports on developing an audio-based monitoring system for detecting swallowing and fluid intake events. Sound signals were collected from 13 subjects using a throat microphone. A framework was introduced, comparing multiple 1D-2D conversion methods to convert audio signals into image data suitable for Convolutional Neural Network (CNN) based classification. Different pretrained CNN models were fine-tuned on our dataset, pairing with different signal lengths. The classification results indicated that Short-time Fourier Transform (STFT) was the optimal method, with an accuracy of over 97% across all signal lengths and Networks. The swallowing detection mechanism achieved 92.45% accuracy and 0.88 F1-score, with the capability to occur 0.35 seconds before the signal peak. These results and robust performance under diverse noise conditions highlight the system's potential for real-time application in real-world scenarios.