Emergency Signal Classification for the Hearing Impaired using Multi-channel Convolutional Neural Network Architecture

Hearing impaired people have to tackle a lot of challenges, particularly during emergencies, making them dependent on others. The presence of emergency situations is mostly comprehended through auditory means. This raises a need for developing such systems that sense emergency sounds and communicate it to the deaf effectively. The present study is conducted to differentiate emergency audio signals from non-emergency situations using Multi-Channel Convolutional Neural Networks (CNN). Various data augmentation techniques have been explored, with particular attention to the method of Mixup, in order to improve the performance of the model. The experimental results showed a cross-validation accuracy of 88.28 % and testing accuracy of 88.09 %. To put the model into practical lives of the hearing impaired an android application was developed that made the phone vibrate every time there was an emergency sound. The app could be connected to an android wear device such as a smartwatch that will be with the wearer every time, effectively making them aware of emergency situations.