H.264 is the latest standard for video compression. The H.264 encoder is often regarded as a heavy application for most of the modern SoC (system-on-chip) systems. Thus many of them implement H.264 with ASICs. Although the ASIC-based approach can benefit from low power consumption, area minimization, and high performance, it suffers from the lack of flexibility which increases NRE cost and time-to-market. As an alternative solution, ASIP-based designs have been suggested to alleviate the problems of ASIC approaches. However, ASIP approach also suffers from its insufficient computing power to run H.264. In this paper, we propose an ASIP solution which partially incorporates ASIC-style complex instruction to boost up the performance. Our architecture shows the similar area cost and performance as compared to full ASIC-based designs, but exhibits 52% better performance as compared to TI DSP. It also runs 9 times faster for fully optimized x264 on Pentium D.