Simple unit-pruning with gain-changing training

In this paper, a novel scheme for pruning the units within a neural network is proposed. The proposed scheme consists of a simple unit-pruning algorithm augmented by a new training algorithm called gain-changing training. In the gain-changing training, the gain of each unit is changed in order that the functions are concentrated on fewer units, i.e., some units play important roles and others negligible roles. Experiments with neural filters (NFs) to reduce noise from natural and medical images were performed. The experimental results demonstrated that the performance of the proposed scheme is superior to those of the conventional methods including the optimal brain damage method (OBD): the proposed scheme resulted in smaller networks; the NFs obtained by the proposed scheme achieved higher performance and generalization ability.