Local gain adaptation in stochastic gradient descent
Schraudolph, N.N.
Artificial Neural Networks, 1999. ICANN 99. Ninth International Conference on (Conf. Publ. No. 470)
Volume 2, Issue , 1999 Page(s):569 - 574 vol.2
Digital Object Identifier  
Summary:Gain adaptation algorithms for neural networks typically adjust learning rates by monitoring the correlation between successive gradients. Here we discuss the limitations of this approach, and develop an alternative by extending Sutton's work on linear systems (1992) to the general, nonlinear case. The resulting online algorithms are computationally little more expensive than other acceleration techniques, do not assume statistical independence between successive training patterns, and do not require an arbitrary smoothing parameter. In our benchmark experiments, they consistently outperform other acceleration methods, and show remarkable robustness when faced with non i.i.d. sampling of the input space

» View citation and abstract

Log in by entering your IEEE Web Account Username and Password.

IEEE Communications Society members: If you subscribe to the IEEE Electronic Periodicals Package or IEEE Electronic Periodicals Package Plus, you must access your subscription at www.comsoc.org.

Check with your librarian, information professional, or system manager to determine if you need to log in. Please complete the online Technical Support Form if you need assistance.

Select the Purchase History link to access the document. You will have 5 Days after purchase to access the Full Text PDF. Please complete the online Technical Support Form if you need assistance.

• Search and access Abstract records free of charge
• Register for table of contents alerts
• Purchase Full Text PDF documents

» Learn more about subscription options or how to become an IEEE Member.