A movable image-based rendering system and its application to multiview audio-visual conferencing

Image-based rendering (IBR) is an emerging technology for rendering photo-realistic views of scenes from a collection of densely sampled images or videos. It provides a framework for developing revolutionary virtual reality and immersive viewing systems. This paper studies the design of a movable image-based rendering system based on a class of dynamic representations called plenoptic videos. It is constructed by mounting a linear array of 8 video cameras on an electrically controllable wheel chair with its motion being controllable manually or remotely through wireless LAN by means of additional hardware circuitry. We also developed a real-time object tracking algorithm and utilize the motion information computed to adjust continuously the azimuth or rotation angle of the movable IBR system in order to cope with a given moving object. Due to the motion of the wheel chair, videos may appear shaky and video stabilization technique is proposed to overcome this problem. The system can be used in a multiview audio-visual conferencing via a multiview TV display. Through this pilot study, we hope to develop a framework for designing movable IBR systems with improved viewing freedom and ability to cope with moving object in large environment.