Adaptive gain control and time warp for enhanced speech intelligibility under reverberation

Moderate and severe reverberation reduce speech intelligibility as a result of the overlap-masking effect, which constitutes the simultaneous observation of multiple delayed and attenuated copies of the speech signal. Recent progress has been made in ameliorating the degradation in intelligibility by adaptively controlling the signal gain as a function of both the signal statistics and the properties of the environment. While the intelligibility gain is at present small, it is significant and serves as a clear indication that reduced masking in non-stationary portions of the signal, under appropriate smoothness constraints, correlates well with an increase in intelligibility. A multi-modal modification framework is expected to improve performance further by i) introducing additional means to reduce masking in designated portions of the speech signal and ii) reducing signal distortion introduced by the use of a single modification modality. In particular we consider the combination of adaptive gain control and local time-warp dependent on the gain modification. A listening test shows that the proposed approach outperforms linear time warp. Objective validation results provide additional insight into the specifics of the proposed multi-modal modification approach.