Input pattern encoding through generalized adaptive search

In a neural network approach to a sequence prediction problem such as Chinese character prediction, if an orthogonal set is used to encode the Chinese characters, there will be more than 6000 units in the input layer. The authors demonstrate that the number of units in the input layer can be greatly reduced with proper encoding. A neural network maps a group of input vectors to a group of target vectors. It generalizes the responses for inputs that are similar to the inputs on which it has been trained. With this similarity property, if the input pattern vectors are encoded according to the interrelationship among the target patterns, the network may behave better, and fewer units will be needed in the input layer. The authors present such an input pattern encoding method for a neural network with recurrent connections. A modified genetic algorithm was used to do a generalized adaptive search for a good encoding.<>