Forecasting by density shaping using neural networks

An estimate of the probability density function of a random vector is obtained by maximizing the output entropy of a feedforward network of sigmoidal units with respect to the input weights. A normalized version of the sigmoidal transfer function simplifies the algorithm considerably and leads to a maximum entropy estimate of the input density under a certain model. Newton's method, applied to the estimated density, yields a recursive estimator for a random variable or a random sequence. A constrained connectivity structure yields a linear estimator, which is particularly suitable for "real time" prediction.