Residual modeling in music analysis-synthesis

In analysis-synthesis of musical sounds based on a sinusoidal model, the difference between the original signal and the synthesized signal, termed the residual, is typically a broadband noise process. It contains such musical phenomena as flute breath noise or violin bow noise. Synthesis without such "noise" tends to sound artificial; it is desirable to improve the synthesis realism by modeling the residual in such a way that it can be reinjected in the synthesized signal. This paper deals with a model of noise perception based on the equivalent rectangular bands (ERBs) of the auditory system. Since a broadband noise is perceptually well-represented by the time-varying energy in each of these frequency bands, the residual is parametrized in terms of these energies in the proposed model. An application of the model to music synthesis based on the inverse fast Fourier transform (FFT) is described in detail.