Deterministic coordinate descent algorithms for smooth convex optimization

In this paper, we study a deterministic coordinate descent algorithm for constrained smooth convex optimization, where each iteration involves updating a block of the decision vector via a block coordinate-wise gradient projection step. We establish the asymptotic convergence of the algorithm under a minimal assumption that all the blocks would be updated infinitely many times. Moreover, we derive an O(1/k) rate of convergence to optimality for the algorithm, provided that each block is updated at least once during any time interval of given length. The algorithm generalizes the cyclic block coordinate gradient projection method in terms of the block selection rule and the step-size choice, and is shown to have commensurate convergence rate guarantees. The convergence rate of the algorithm can further be accelerated to O(l/k
²
) by integrating the algorithm into a multi-step optimal gradient method when the problem is unconstrained. Finally, extensive simulations illustrate the efficiency and scalability of the deterministic coordinate descent algorithm and its accelerated variant.