Real-Time Speech Emotion Recognition Using Hybrid Transformer Networks for Mental Health Monitoring

This study proposes a hybrid method to multi dialect speech recognition using the transfer learning with transformer-based architecture. The primary objective of this method is to enhance the automatic speech recognition (ASR) across the different dialectal variations of specific language. The transfer learning methods are utilized to modify the base model that was trained on high resource language to manage the multiple dialects with limited labelled data for each dialect. This is achieved by using pre trained transformer methods such as Wav2Vec2.0 and Whisper. An approach which employs dialect specific embeddings is presented. It enables the method to capture data and differentiate between prosodic elements and phonetic that are distinctive to a variety of dialects. Data augmentation technique are utilized in order to overcome the limited availability of labelled data for low resource dialects. These methods are utilized to increase the efficiency of training set and enhance the robustness of the method. Evaluation of the method is carried out using multi dialect speech datasets which include Arabic MGB-3 and common voice. These datasets represent a variety of dialects that are spoken in various linguistic regions. The result of this study indicates that the transformer-based method is effective in managing difficulties of dialectal variances while the reliability is reduced on different datasets. Different real-world applications are provided by the proposed method for improving voice enabled applications. Furthermore, this method leads the way for the development of ASR method that are more accurate and independent in locations that have a significant linguistic variation.