Loading [a11y]/accessibility-menu.js
Improving gradient-based learning algorithms for large scale feedforward networks | IEEE Conference Publication | IEEE Xplore

Improving gradient-based learning algorithms for large scale feedforward networks


Abstract:

Large scale neural networks have many hundreds or thousands of parameters (weights and biases) to learn, and as a result tend to have very long training times. Small scal...Show More

Abstract:

Large scale neural networks have many hundreds or thousands of parameters (weights and biases) to learn, and as a result tend to have very long training times. Small scale networks can be trained quickly by using second-order information, but these fail for large architectures due to high computational cost. Other approaches employ local search strategies, which also add to the computational cost. In this paper we present a simple method, based on opposite transfer functions which greatly improve the convergence rate and accuracy of gradient-based learning algorithms. We use two variants of the backpropagation algorithm and common benchmark data to highlight the improvements. We find statistically significant improvements in both convergence speed and accuracy.
Date of Conference: 14-19 June 2009
Date Added to IEEE Xplore: 31 July 2009
ISBN Information:

ISSN Information:

Conference Location: Atlanta, GA, USA

References

References is not available for this document.