Logistic Regression Based on Bat-Inspired Algorithm with Gaussian Initialization

Logistic regression is a supervised statistical learning method in the field of statistics and machine learning nowadays. The core idea of traditional logistic regression algorithm is to use gradient descent method to find the optimal parameter of loss function with certain convexity. As far as this kind of nonlinear problem is concerned, gradient descent (GD) method or stochastic gradient descent (SGD) method in linear search is generally used, however, there are some shortcomings of GD and SGD in dealing with the optimal parameter finding issue for loss function with certain convexity. Therefore, it is of great significance to study such issue that traditional logistic regression cannot properly handle. As a matter of fact, in the process of solving the optimal parameter of convex loss function, bat heuristic algorithm can be used to replace the traditional penalty function method, moreover, bat heuristic algorithm is a gradient independent, swarm intelligence algorithm, which can avoid falling into local minimum trap by dynamically controlling parameters. In this paper, batbased logistic regression algorithm has been proposed so as to deal with the multi-data classification problem, the proposed method can not only handle data without gradient information, but handle data without differentiable node transfer function as well, and its effectiveness has been strictly verified via numerical analysis.