H.264/AVC plays an important role in the video compression standard, it is better than previous video standards in the compression ration and the image quality. Motion estimation is one of the core designs of H.264 video coding, it basically includes the motion vector at quarter pixel resolution with variable block sizes and multiple reference frames, it mainly improves the image quality and provides more accurate predictions, however, because of these features, the computation load and complexity of motion estimation increase significantly. In this paper, a VLSI architecture for variable block size motion estimation with three step search algorithm is proposed. In order to improve the throughput, parallel architecture is adopted and the processing elements also allow the sums of absolute differences of larger blocks to be computed by using the results derived for 4×4 blocks. The proposed method can obtain the motion vectors of different block size. Compared to the previous architectures for variable block size, our architecture can reduce the computational complexity.