MhAFN: A Multi-Head Attention-Based Fusion Network for Personality Prediction from Multimodal Text and Audio Data

This proposed study aims to design a novel Deep Learning mechanism for personality prediction through text and audio data. Initially, Unicode normalization is used to preprocess the text data. After the text features are extracted using the XLM-Robustly Optimized BERT Approach, text generation is performed using the Bi-directional Gated Recurrent Unit Bi-GRU model. Finally, the obtained text is converted into voice using a text-to-speech conversion process. Similarly, for the audio data, the audio signal is pre-processed by wavelet denoising transformer methods. Then, the audio features are extracted using the Light Attention Connected VGGish (LACVGGish) Network model. Finally, both extracted features from text and audio are fused by using a multi-head attention based fusion network (MhAFN) model. The experimental result of the proposed model is evaluated by using a real time dataset. A high accuracy performance is achieved on text and audio data at values of 98.52% and 98.85%, respectively.