Search patterns and the center-biased characteristics of motion vector distribution (MVD) have a large impact on both searching speed and quality of block motion estimation. We propose a novel algorithm using two cross-shaped search patterns as the first two initial steps and large/small diamond-shaped patterns as the subsequent steps for fast block motion estimation (BME). The first small cross-shaped pattern is to fit the cross-center-biased MVD characteristics of the real-world sequences by evaluating the 5 relatively higher probable candidates located as a cross-shaped pattern at the search-grid center. The proposed small-cross-diamond search algorithm (SCDS) employs a halfway-stop technique and could find small motion vectors with much fewer points than the diamond search algorithm (DS) while maintains similar or even better quality. The speedup improvement of SCDS over DS can be up to 146%, i.e. 2.46 times faster than DS. Simulations show that SCDS is much more robust, provides faster searching speed and smaller distortions than other fast algorithms, typically very suitable for videoconferencing applications.