By Topic

Role of activation function on hidden units for sample recording in three-layer neural networks

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

1 Author(s)

It is shown that the k hidden units with asymptotic activation function are able to transfer any given k+1 different inputs to linearly independent GHUVs (generated hidden unit vectors) by properly setting weights and thresholds. The number of hidden units with the LIT (linearly independent transformation) capability for the polynomial activation function is limited by the order of polynomials. For analytic asymptotic activation functions and given different inputs, the LIT is a generic capability and a probability 1 capability in setting weights and thresholds randomly. It is a generic and a probability 1 property for any random input if the weight and threshold setting has LIT capability for some k+1 inputs. For three-layer nets with k hidden units, in which the activation function is asymptotic and the output layer is without activation function, they are sufficient to record k+1 arbitrary real samples. It is probability 0 to record k+2 random real samples if the activation is a unit step function. This is true for the sigmoid function in the case of associative memory. These conclusions lead to a scheme for understanding associative memory in the three-layer networks

Published in:

Neural Networks, 1990., 1990 IJCNN International Joint Conference on

Date of Conference:

17-21 June 1990