Speeding up MLP execution by approximating neural network activation functions

At present the multilayer perceptron model (MLP) is, without doubt, the most used neural network for applications so it is important to design and test methods to improve MLP efficiency at run time. This paper analyzes the error introduced by a simple but effective method to cut down execution time for MLP networks dealing with a sequential input, this is a very common case, including all kinds of temporal processing, like speech, video, and in general signals varying in time. The technique requires neither specialized hardware nor large quantities of additional memory and is based on the ubiquitous idea of difference transmission, widely used in signal coding. It requires the introduction of a sort of quantization of the unit activation function; this causes an error which is analyzed in this paper from a theoretical point of view.