By Topic

Determine the Irrelevance of Hidden Unit from the Validation Set

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

The purchase and pricing options are temporarily unavailable. Please try again later.
2 Author(s)
Jearanaitanakij, K. ; King Mongkut''s Inst. of Technol., Bangkok ; Pinngern, O.

This paper proposes a method to determine the irrelevance of the hidden unit in the artificial neural network. Unlike other approaches, we calculate the sensitivity of the hidden unit from the validation set, instead of the training set. The advantage of using the validation set to calculate the sensitivity is that we never overestimate the relevance of hidden unit. In other words, we always remove the unit that has the least effect on the validation set error. As a result, the pruned neural network has the highest generalization when compared with other choices of removals. Our sensitivity is based on the activation difference of the output unit. This activation difference is the gap between the activation of output units when a particular hidden unit is present and when it is removed. We have applied our technique to two standard benchmark problems. The experimental results show that the proposed technique can correctly determine the least irrelevant hidden unit

Published in:

Communications and Information Technologies, 2006. ISCIT '06. International Symposium on

Date of Conference:

Oct. 18 2006-Sept. 20 2006