Generalization by weight-elimination applied to currency exchange rate prediction

Inspired by the information theoretic idea of minimum description length, the authors add a term to the backpropagation cost function that penalizes network complexity. The authors give the details of the procedure, called weight-elimination, describe its dynamics, and clarify the meaning of the parameters involved. From a Bayes perspective, the complexity term can be usefully interpreted as an assumption about prior distribution of the weights. The goal is to find a net that has the lowest complexity while fitting the data adequately. The complexity is dominated by the number of bits needed to encode the weights. It is roughly proportional to the number of weights times the number of bits per weight. The authors focus on the procedure of weight-elimination that tries to find a net with the smallest number of weights. They compare it with a second approach that tries to minimize the number of bits per weight, thereby creating a net that is not too dependent on the precise values of its weights. The authors use this procedure to predict currency exchange rates.<
>