Real-Time Sound Recognition System for Human Care Robot Considering Custom Sound Events

In real-life situations where human care robots are deployed, there are custom sound events whose acoustic characteristics change depending on the user’s choice unlike general sound events so that the human care robots cannot recognize custom sound events correctly in a conventional way. To solve this critical problem, a real-time sound event recognition system with customization process is proposed. The human care robot collects custom sound samples of a specific user and customizes a sound event recognition model. The overfitting-based customized model shows the best recognition performance by improving F-scores by 66.4% on average compared to the conventional recognition model. After the customization process, the human care robot performs a real-time sound recognition by consistently streaming robot’s real-time microphone signals into the overfitting-based customized SER model. In this process, an optimized overlap is applied on subsequent audio inputs on SER to achieve sufficiently fast response and robust performance. As a pilot test of the human care robot implemented in actual environment, the real-time sound recognition system shows the best average F-score of 0.982 with 75% overlap for sound events including custom sounds. This pilot test result confirms that the real-time sound recognition system with customization process can be successfully applied to human care robots to respond to the custom sounds.