Real-Time Signal Estimation From Modified Short-Time Fourier Transform Magnitude Spectra

An algorithm for estimating signals from short-time magnitude spectra is introduced offering a significant improvement in quality and efficiency over current methods. The key issue is how to invert a sequence of overlapping magnitude spectra (a ldquospectrogramrdquo) containing no phase information to generate a real-valued signal free of audible artifacts. Also important is that the algorithm performs in real-time, both structurally and computationally. In the context of spectrogram inversion, structurally real-time means that the audio signal at any given point in time only depends on transform frames at local or prior points in time. Computationally, real-time means that the algorithm is efficient enough to run in less time than the reconstructed audio takes to play on the available hardware. The spectrogram inversion algorithm is parameterized to allow tradeoffs between computational demands and the quality of the signal reconstruction. The algorithm is applied to audio time-scale and pitch modification and compared to classical algorithms for these tasks on a variety of signal types including both monophonic and polyphonic audio signals such as speech and music.

I. Introduction

Magnitude spectra and their time sequences in the form of spectrograms are widely used for time-frequency representations of audio signals such as speech and music. The magnitude spectrum of a discrete-time signal is typically obtained from the short-time Fourier transform (STFT), which is defined as $$X(mS,\varpi)=\sum_{n=-\infty}^{\infty}x(n)w(n-mS)e^{-j\varpi n}\eqno{\hbox{(1)}}$$ where is the analysis window, is the analysis step size, and is the index of the frames of the STFT. The complex-valued STFT is a complete and reversible time-frequency representation. The time-domain signal is uniquely determined by its STFT representation and vice versa. Using the STFT, the short-time Fourier transform magnitude (STFTM) spectrum of is $$\left\vert X(mS,\varpi)\right\vert=\left\vert\sum_{n=-\infty}^{\infty}x(n)w(n-mS)e^{-j\varpi n}\right\vert.\eqno{\hbox{(2)}}$$