Enhancement of speech intelligibility under noisy reverberant conditions based on modulation spectrum concept

This study focuses on identifying effective features for controlling speech to increase speech intelligibility under adverse conditions. Previous methods either reduce noise and reverberation throughout speech presentation or enhance speech before presenting it by controlling its intensity and/or spectral properties to increase intelligibility. Among them, a method based on modulation transfer function theory, in which the environmental effects are inverted to anticipate attenuation of the modulation spectrum of speech, shows excellent potential due to its systematic and explicit derivation of intelligibility enhancement against environmental smears. However, directly obtaining that inversion requires estimating the modulation transfer function. The estimate seems complicated and tolerant under realistic variable conditions. This study takes a different approach: analyzing the relations of smeared modulation spectra by the environments for intelligibility to extract effective modifying features. First, we conduct listening tests for intelligibility in noise with different types of enhanced speech. Next, we extract acoustic and modulation frequency components in the smeared modulation spectra by noise showing high correlation with intelligibility scores. Finally, we examine the intelligibility benefits of modifying these components by performing listening tests. The results show that these components effectively increase intelligibility by at most 20%, which demonstrates that our concept is valid.