Accelerated Optimization with Orthogonality Constraints (Journal of Computational Mathematics, 2020)
We develop a generalization of Nesterov's accelerated gradient descent method whichis designed to deal with orthogonality constraints. To demonstrate the effectiveness ofour method, we perform numerical experiments which demonstrate that the number ofiterations scales with the square root of the condition number, and also compare withexisting state-of-the-art quasi-Newton methods on the Stiefel manifold. Our experimentsshow that our method outperforms existing state-of-the-art quasi-Newton methods on somelarge, ill-conditioned problems.