Compact Near-Real-Time Anger Detection System Through Voice Analysis Implemented on ESP32 Microcontroller

This article presents the development of a compact, near-real-time anger detection system through voice analysis, implemented on an ESP32 microcontroller. The system is designed to address domestic violence scenarios in which anger often precedes aggression. Due to the computational constraints of embedded devices, we propose an optimized convolutional neural network (CNN) based on VGG16, called MiniVGG16, which reduces the number of parameters while maintaining an 80% accuracy in anger recognition. The system processes audio signals in near-real-time, extracting mel-frequency cepstral coefficients (MFCCs) from speech data. These features are then fed into the MiniVGG16 model, which has been optimized using TensorFlow Lite Micro (TFLM) for efficient inference on the ESP32. The entire pipeline, including audio sampling, feature extraction, and classification, is executed in 676.4 ms, enabling practical deployment in near-real-world applications. The system’s small memory usage and real-time processing capability make it a cost-effective and accessible tool for emotion monitoring in home environments, mental healthcare scenarios, and security applications. Future improvements include noise filtering and data augmentation to improve classification performance under diverse acoustic conditions.