Imposing Correlation Structures for Deep Binaural Spatio-Temporal Wiener Filtering

To improve speech quality and intelligibility in environments with noise and interfering sounds, binaural speech enhancement algorithms use the microphone signals from both the left and the right hearing device to generate an enhanced output signal for each ear. As a multi-frame extension of the binaural multi-channel Wiener filter, in this paper we consider the binaural spatio-temporal Wiener filter (STWF) in the short-time Fourier transform domain, which requires estimates of the highly time-varying spatio-temporal correlations of the speech and interference components. To this end, the binaural STWF is embedded into an end-to-end supervised learning framework, where temporal convolutional networks estimate the required quantities, i.e., the inverse spatio-temporal correlation matrices of the interference component and the spatio-temporal correlation vectors and power spectral densities of the speech components. In this paper, we impose spatio-temporal correlation structure on these quantities and relate them between the left and the right hearing device, aiming to reduce computational complexity while maintaining speech enhancement and interaural cue preservation performance. Assuming that the spatial correlation of the speech component is stationary over a small number of frames, we propose to decompose the spatio-temporal correlation vectors as the Kronecker product of a relative transfer function vector and a temporal correlation vector, either considering a global reference microphone or a reference microphone for each hearing device. In addition, we consider a deep bilateral STWF by neglecting the spatio-temporal correlations of the speech and interference components between both devices. The imposed spatio-temporal correlation structures greatly differ in the number of parameters that need to be estimated. The performance of causal versions of the deep binaural and bilateral STWF algorithms is evaluated based on both simulated and measured binaural room impulse responses (BRIRs) as well as diverse speech and noise sources. The simulation results demonstrate that the proposed spatio-temporal correlation structures significantly reduce the computational complexity of the binaural STWF while yielding a similar speech enhancement and interaural cue preservation performance compared to not imposing any spatio-temporal correlation structure. Furthermore, the results confirm that the deep binaural STWF outperforms the binaural Conv-TasNet algorithm as well as an algorithm that directly estimates the binaural multi-frame filter coefficients, while approaching the performance of the non-causal binaural complex convolutional transformer network (BCCTN) algorithm.