A method for stochastic L-BFGS optimization

In the large-scale machine learning, it is very important to effectively train machine learning model. Stochastic gradient descent and its variants have achieved good performance in model training. Recently there have been the stochastic L-BFGS method, which has better efficiency of the training compared to stochastic gradient descent. However, it is difficult to find a suitable step selection method for the method. We present a method for stochastic L-BFGS optimization called Aadaquasi and prove its global convergence for convex optimization problem. Using the search direction and its history, it employs the new update rule of the step length and expands scope of step length so that the stochastic L-BFGS algorithm is more robust. Experiments on webspam classification model show that the method can guarantee the global convergence and has a better convergence efficiency.