Orthogonality is Better: Auxiliary Problems in ASO Algorithm

We propose a principle called orthogonality for Auxiliary problems (APs) selection in Alternating structure optimization (ASO) algorithm. Both theoretical analyses and experimental results indicate the following conclusions. If the weight matrices of different types of APs are orthogonal or approximately orthogonal, their multi-combinations perform better than or equal to any components. Moreover, as long as the ratios of their components are appropriate, even if the total amounts of APs are fixed, the multi-combinations still perform better than or equal to any components. In short, the principle of orthogonality holds.