Amari, S.
Douglas, S.C.
RIKEN, Inst. of Phys. & Chem. Res., Saitama ;
This paper appears in: Acoustics, Speech and Signal Processing, 1998. Proceedings of the 1998 IEEE International Conference on
Publication Date: 12-15 May 1998
Volume: 2,
On page(s): 1213-1216 vol.2
Meeting Date: 05/12/1998 - 05/15/1998
Location: Seattle, WA, USA
ISSN: 1520-6149
ISBN: 0-7803-4428-6
References Cited: 5
INSPEC Accession Number: 6040105
Digital Object Identifier: 10.1109/ICASSP.1998.675489
Posted online: 2002-08-06 21:42:08.0
Abstract
Gradient adaptation is a useful technique for adjusting a set of
parameters to minimize a cost function. While often easy to implement,
the convergence speed of gradient adaptation can be slow when the slope
of the cost function varies widely for small changes in the parameters.
In this paper, we outline an alternative technique, termed natural
gradient adaptation, that overcomes the poor convergence properties of
gradient adaptation in many cases. The natural gradient is based on
differential geometry and employs knowledge of the Riemannian structure
of the parameter space to adjust the gradient search direction. Unlike
Newton's method, natural gradient adaptation does not assume a
locally-quadratic cost function. Moreover, for maximum likelihood
estimation tasks, natural gradient adaptation is asymptotically
Fisher-efficient. A simple example illustrates the desirable properties
of natural gradient adaptation
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
