In linear multi-input multi-output channel systems, reduction process is employed to reduce the computational cost of sphere decoding (SD) algorithm. Usually a reduction process includes not only permutations but also unimodular transformations. However, owing to the box-constraint, the current reduction strategies are limited to only permutations, which makes SD algorithm still very time-consuming. In this study a theoretical complexity analysis on SD algorithm is first proposed to show what kind of criteria a reduction process should pursue. Then a new reduction strategy which combines permutations with unimodular transformations is presented to obtain a better reduced detection problem. Simulation results showed that this new reduction strategy can make SD algorithm much more efficient than those reduction strategies with only permutations.