By Topic

Hidden neuron pruning for multilayer perceptrons using a sensitivity measure

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Yeung, D.S. ; Dept. of Comput., Hong Kong Polytech. Univ., Kowloon, China ; Xiao-Qin Zeng

In the design of neural networks, the way to choose the proper size of a network for a given task is an important and difficult problem that still deserves further exploration. One popular approach for tackling this problem is to first use an oversized network and then pruning it to a smaller size so as to achieve less computational complexity and better performance in generalization. This paper presents a pruning technique, via a quantified sensitivity measure, to remove as many neurons as possible, those with the least relevance, from the hidden layers of a multilayer perceptron (MLP). The sensitivity of an individual neuron is defined as the expectation of its output deviation due to expected input deviation with respect to the overall inputs from a continuous interval. The relevance of a neuron is defined as the multiplication of its sensitivity value by the summation of its outgoing weights. The basic idea is to iteratively train the network according to a certain performance criterion and then remove the neurons with the lowest relevance values. The pruning technique is novel in its quantified sensitivity measure. Computer simulations demonstrate that it works well.

Published in:

Machine Learning and Cybernetics, 2002. Proceedings. 2002 International Conference on  (Volume:4 )

Date of Conference:

4-5 Nov. 2002