To realize fast and robust digital image stabilization, in this paper, we propose a new optimum motion estimation algorithm under various illuminations, occlusion and blooming. In this algorithm, we expand our original fast template matching method named HDTM (hierarchical distributed template matching). First, only useful reference blocks that are indispensable for accurate motion estimation are selected with its reliability and consistency on pose estimation. Next, using the LMedS method, motion vectors of these blocks are segmented to some groups and only reliable ones are used for whole motion estimation. With experimental results, we could see small errors less than ±0.1 pixels, ±0.1 degrees and ±0.1% scales were achieved under various kinds of disturbance. And the processing time was 11 msec with PII-300 MHz CPU in typical case. It is enough to realize real-time processing for embedded use of image stabilization.