Gradient Descent and Newton's Method with Backtracking Line Search in Linear Regression

The accuracy and efficiency of optimization algorithms remain an essential challenge in supervised machine learning. We used gradient descent and Newton's method, along with backtracking line search, to determine optimal parameter values in linear regression. The obtained results for both models are accurate and are close to the straightforward solutions using the least squares method. Newton's method with backtracking line search performs better with less iteration steps and execution time, suggesting that this method has a higher tolerance for controlling and selecting learning rate alpha. Optimization algorithms with backtracking line search method can be applied to solve diverse regression problems with high precision and efficiency.