On the Convergence and Implementation of High-Order Primal Dual Algorithms for Affine-Constrained Convex Optimization

Recently, Nesterov proposed an implementable tensor method for unconstrained convex optimization problems, and showed that the p-th order algorithm converges at a rate of $\mathcal{O}\left( {{\varepsilon ^{ - \frac{1}{{p + 1}}}}} \right)$, where p is the degree of Taylor approximation. In this paper, we extend this high-order method to affine constrained convex optimization problems. Under the same assumption for the objective function, we show that the p-th order primal-dual algorithm converges. Moreover, we introduce two inner-loop algorithms for the practical implementation of the proposed high-order primal-dual method. Extensive numerical experiments show that, as long as the step size of dual gradient ascent updates is not too small, the proposed high-order primal-dual method outperforms vanilla primaldual methods with low-order primal updates. However, if using optimized number of inner loops without accurately solving the inner problem in primal updates, algorithms with 2nd- and 3rd-order primal updates yield comparable results.