Neural information processor

We propose a neural information processor composed of an information filter and compressor. The information processor is introduced to unify information maximization and minimisation methods so far independently developed to control internal representations. The information filter and compressor of the information processor are simultaneously or sequentially applied. First, the information filter aims to decrease information contained in hidden units as much as possible and to capture information relevant to learning. The information compressor is used to compress information processed by the information minimisers into a small number of hidden units. To examine the performance of the information processor, it was applied to language acquisition problems. Experimental results confirmed that information can be maximised and minimized, and we could obtain simplified internal representations from redundant networks. In addition, generalization performance could significantly be improved. Thus, this neural information processor can be used not only to improve generalization and interpretation of internal representations but also to model human intelligent information processing in general.