Hilbert-Huang-Transform Based Features for Accent Classification of Non-Native English Speakers

The development of accurate accent-recognition technologies is important towards high-performance Automatic Speech Recognition (ASR) through the use of accent-specific models. Usually deep-learning models are employed to perform ASR and we intend to enhance such models by accurately capturing the accent-related idiosyncrasies of non-native English speakers. The typical features used for the deep-learning networks, in ASR, are Mel-Frequency Cepstral Coefficients (MFCCs) and Mel-Spectrogram of the audio samples. However, novel features based on the Hilbert-Huang Transform (HHT), an effective non-linear signal analysis technique can aid or even replace typical features used in Automatic Speech Recognition. In this paper, we seek to apply these features in the task of accent recognition for non-native English speakers.We propose a new novel input feature, the Hilbert Mel-Spectrogram, a logarithmic frequency representation of the signal derived using the HHT. The use of HHT creates an input feature free of the traditional limitations of Fourier analysis, namely limited frequency resolution, spectral energy leakage, and harmonic artifacts. A four-stage Convolution Neural Network (CNN) is used to model the features of accented speech samples spanning 5 native languages. Our results show that the models with the Hilbert-Mel-Spectrogram-based features outperform their Fourier-based counterparts for a wide range of datasets.