In greedy PURSUIT of new directions: (Nearly) Orthogonal Matching Pursuit by directional optimisation

Matching Pursuit and orthogonal Matching Pursuit are greedy algorithms used to obtain sparse signal approximations. Orthogonal Matching Pursuit is known to offer better performance, but Matching Pursuit allows more efficient implementations. In this paper we propose novel greedy Pursuit algorithms based on directional updates. Using a conjugate direction, the algorithm becomes a novel implementation of orthogonal Matching Pursuit, with computational requirements similar to current implementations based on QR factorisation. A significant reduction in memory requirements and computational complexity can be achieved by approximating the conjugate direction. Further computational savings can be made by using a steepest descent direction. The two resulting algorithms are then comparable to Matching Pursuit in their computational requirements, their performance is however shown to be closer to that of orthogonal Matching Pursuit with the (slightly slower) approximate conjugate direction based approach outperforming the gradient descent method.