Stabilized Barzilai-Borwein Method (Journal of Computational Mathematics, 2021)
The Barzilai-Borwein (BB) method is a popular and efficient tool for solving large-scaleunconstrained optimization problems. Its search direction is the same as the steepestdescent (Cauchy) method, but its step size rule is different. Owing to this, it convergesmuch faster than the Cauchy method. A feature of the BB method is that it may generatetoo long steps, which throw the iterates too far away from the solution. Moreover, itmay not converge, even when the objective function is strongly convex. In this paper, astabilization technique is introduced. It consists in bounding the distance between eachpair of successive iterates, which often allows for decreasing the number of BB iterations.When the BB method does not converge, our simple modification of this method makesit convergent. For strongly convex functions with Lipschits gradients, we prove its globalconvergence, despite the fact that no line search is involved, and only gradient values areused. Since the number of stabilization steps is proved to be finite, the stabilized versioninherits the fast local convergence of the BB method. The presented results of extensivenumerical experiments show that our stabilization technique often allows the BB methodto solve problems in a fewer iterations, or even to solve problems where the latter fails.