Algorithm and architecture of video segmentation hardware system with a programmable PE array

Video segmentation is a key unit in content-based video encoding systems, such as MPEG-4. Existing algorithms are too complex for real-time applications, and hardware implementation is infeasible because of the global and irregular operations. In this paper, a hardware system for video segmentation is proposed from algorithm level to hardware architecture level. A hardware-oriented algorithm is first proposed to generate accurate object masks with local pixel operations and morphological operations, which are suitable for hardware implementation. After that, the hardware architecture is designed based on partial-result-reuse architecture and programmable morphology PE array architecture, which can achieve both high flexibility and throughput. A prototype chip is implemented to achieve the processing speed of 30 QCIF frames per second and 7,680 morphological operations per second at 26 MHz. It also shows the hardware cost is small, and the proposed video segmentation hardware system is suitable to be integrated into any content-based video encoding systems.