30 years of adaptive neural networks: perceptron, Madaline, andbackpropagation
Widrow, B.
Lehr, M.A.
Dept. of Electr. Eng., Stanford Univ., CA;
This paper appears in: Proceedings of the IEEE
Publication Date: Sep 1990
Volume: 78,
Issue: 9
On page(s): 1415-1442
ISSN: 0018-9219
References Cited: 133
CODEN: IEEPAD
INSPEC Accession Number: 3796897
Digital Object Identifier: 10.1109/5.58323
Current Version Published: 2002-08-06
Abstract
Fundamental developments in feedforward artificial neural networks
from the past thirty years are reviewed. The history, origination,
operating characteristics, and basic theory of several supervised
neural-network training algorithms (including the perceptron rule, the
least-mean-square algorithm, three Madaline rules, and the
backpropagation technique) are described. The concept underlying these
iterative adaptation algorithms is the minimal disturbance principle,
which suggests that during training it is advisable to inject new
information into a network in a manner that disturbs stored information
to the smallest extent possible. The two principal kinds of online rules
that have developed for altering the weights of a network are examined
for both single-threshold elements and multielement networks. They are
error-correction rules, which alter the weights of a network to correct
error in the output response to the present input pattern, and gradient
rules, which alter the weights of a network during each pattern
presentation by gradient descent with the objective of reducing
mean-square error (averaged over all training patterns)
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
