On-Device MFCC-CNN Voice Recognition System with ESP-32 and Web-Based Application

The ever-increasing need for intelligence applications capable of assisting human daily life has become a critical topic in current human everyday life. As the primary communication source, deep learning (DL) based speech recognition technique has spread in significant areas in education, transportation, and smart-home system. However, this technique has challenges as it requires numerous datasets and is difficult to implement on a low-cost and low-power system on a chip microcontroller, i.e., ESP-32 or Arduino. With its implementation in controlling the household device, a speech recognition system based on a convolutional neural network (CNN) and MFCC algorithm is proposed in this paper. Firstly, the audio datasets containing specified instructions (in Indonesian) are created, and their features are extracted using MFCC algorithms. Then, the CNN model trains the MFCC vector coefficients and saves them in the Hierarchical Data Format 5 File (h5). This CNN model is stored in the back-end part of the web application and accessed using Flask Python. The audio input (using. wav file format) is sent to the Flask using XMLHttpRequest (XHR) and processed to ESP-32 for controlling the devices. The simulation results, which contain six command words, showed that the selected model achieved 93 % accuracy and 28,39 ms response time.