On information representation in backpropagation classifier networks

Feedforward backpropagation networks have been studied to determine how the external training environment is represented internally. It is shown that for networks trained with simple input-output pattern pairs, the network weights as a whole form strong correlations with the others. Thus, the nets act as correlation-decorrelation memories. It is shown that hidden units function as difference operators, signalling what is unique about certain input patterns compared to the others.<>