Maximizing information about a noisy signal with a single non-linear neuron

For noise-free information maximization, the output signal entropy must be maximized. This is not true for a noisy input: rather, it must be the difference between this entropy and the residual output uncertainty. A definition of information density is introduced, which provides a discrete local measure of bandwidth efficiency. Novel training rules are proposed which enforce a uniformity of this density. This entails a different transfer function from that which follows from the maximization of output entropy alone. It is shown to provide higher information transmission properties on real and synthetic data.