Aliasing-Free Nonlinear Signal Processing Using Implicitly Defined Functions

Digital signal processing relies on the Nyquist-Shannon sampling theorem that applies to and requires a continuous signal with limited bandwidth. However, many systems or networks of signal processing involve nonlinear functions, which could generate new frequency components beyond the original bandwidth and lead to aliasing. Indeed, aliasing-induced shift variance has long been a nuisance and unsolved problem in convolutional neural networks and has recently been found to severely impair the performance of machine learning applications. The same problem exists in other fields such as computational lithography. In this paper, a new method and algorithms are introduced to solve the problem of aliasing induced by nonlinear functions involving operations other than linear convolutions and pointwise multiplications. Said new method and algorithms employ implicitly defined functions that are implemented via iterations of polynomial operations so that aliasing is completely avoided by upsampling signals before polynomial operations and limiting signal spectra before downsampling. Theoretical analyses and exemplary algorithms are presented to implement nonlinear functions commonly used in signal processing networks. In particular, exemplary embodiments and numerical experiments are reported to illustrate and verify aliasing-free operations of Wiener-Padé approximants, which are already universal in their ability to approximate any continuous activation functions to the desired accuracy.