Motion estimation computational complexity reduction with CNN shape segmentation

The MPEG-4 core profile enables object-based coding. A core profile encoder may also have to be able to carry out simple profile frame-based coding. Frame-based coding is, for instance, needed when the object-based functionalities are not used or the target decoder has only simple profile capabilities. cellular nonlinear networks (CNN) offer an effective method to implement the segmentation needed in object-based coding. CNN can be implemented very efficiently with dedicated parallel processor arrays. These parallel processor arrays can integrated with sensor arrays in low-power low bit-rate mobile video encoders. It is shown here that an object-based core profile encoder can have advantages in frame-based simple profile encoding. The intermediate results of a CNN segmentation algorithm can be used in computational complexity reduction of motion estimation (ME). The CNN intermediate results indicate areas of motion within a frame. The areas without motion can be then used to stop the ME or to indicate MPEG-4 skip modes. All block-based ME algorithms can benefit from this knowledge. The intermediate results of the segmentation algorithm are evaluated with the full search and MVFAST ME algorithms. Rate-distortion and computational complexity results are computed with four different reference frame schemes. Considerable improvement in output bits and computational complexity is seen without loss in subjective video quality. The method also has applications in scene change detection.