Speech Emotion Recognition for Psychiatric Care Using Vision Transformers and Predictive Analysis

Speech emotion recognition is the process of ascertaining and identifying feelings in speech so that machines could better understand human emotional states. SER is also being used in customer service for improving the experience of users, security to identify the threats, and healthcare to track emotional wellness. This complex SER system is designed particularly for psychiatric sessions by using the Multiscale Vision Transformer MViTv2 for detecting the emotions of the patient during therapy. The audio signal is captured and then transformed into Mel Spectrograms using Short-Time Fourier Transform (Mel-STFT) of these spectrograms to be fed into MViTv2 to start extracting elaborate emotional features. These features get classified with the aid of Multilayer Perceptron (MLP), which sums up the emotional status of the patient for the session as a whole. Another subsidiary module, LLM, is utilized in predicting the number of the sessions that may be needed for treatment in the future, giving quite valuable predictive insight to the planning of therapy and enabling proper communication between psychiatrists and patients. Such a system demonstrates excellent robust accuracy over multiple datasets and lays the groundwork for further developments, such as video-based emotion recognition, to make psychiatry even more advanced.