Caption detection and removal from video images with complicated background using intelligent inpainting scheme

Effectively finding and removing the captions embedded in video images and restoring the covered background have great significance on video reuse. In this paper, considering the captions color and edge features, we first present a method of caption extraction based on the color edge detector and connected domain features analysis, which can reduce the false detection rate when the gray scale of text is close to background. Meanwhile, the proposed algorithm has stronger robustness and adaptability for captions of different types and countries. Next, after caption detection, we improve a method based on Criminisi algorithm for repairing the text in video image with complex background. We put forward three improvements about this algorithm. First, the fixed size of the template blocks will be changed into the variable ones. Second, we use addition instead of multiplication in computing the priority of repairing blocks. Third, we combine the Euclidean distance between the blocks with the sum of squared differences (SSD) matching criterion. Experimental results demonstrate that our method leads to good results in both detecting the captions and maintaining the video image texture and structure information at the same time.