Video background replacement without a blue screen

An algorithm is presented for replacing video background without a physical blue screen. The method requires pre-recording a background image of the scene without the presence of any foreground objects. Based on the color difference between the pixels in an input frame and their corresponding pixels in the background image, the algorithm computes a probability map which contains the likelihood for each pixel to be classified into the foreground or background. The probability map is further refined using anisotropic diffusion which reduces classification errors without introducing significant artifacts. Finally, based on the refined probability map, a new video is synthesized by feathering the foreground pixels of the input frames onto a new background video or image. The algorithm requires moderate computation resources and has been implemented to run in real time. Experimental results are presented.