By Topic

Object-Based Rendering and 3-D Reconstruction Using a Moveable Image-Based System

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

4 Author(s)
Zhen-Yu Zhu ; Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam, Hong Kong ; Shuai Zhang ; Shing-Chow Chan ; Heung-Yeung Shum

This paper proposes a movable image-based rendering (M-IBR) system for improving the viewing freedom and environmental modeling capability of conventional static IBR systems. The system supports object-based rendering and 3-D reconstruction capability and consists of three main components. 1) An improved video stabilization method to reduce the shaky motion frequently encountered in movable IBR systems. It employs local polynomial regression (LPR) to automatically select an appropriate bandwidth for smoothing the estimated motion. 2) A novel view synthesis algorithm using a new segmentation and mutual-information (MI)-based algorithm for dense depth map estimation, which relies on segmentation, LPRbased depth map smoothing, and MI-based matching algorithm to iteratively estimate the depth map. The method is very flexible and both semiautomatic and automatic segmentation methods can be employed. They rank fourth and sixth, respectively, in the Middlebury comparison of existing depth estimation methods. This allows high-quality renderings of outdoor scenes with improved mobility/freedom to be obtained. 3) A new 3-D reconstruction algorithm that utilizes the sequential structure-from-motion technique and the dense depth maps estimated previously. It relies on a new iterative point cloud refinement algorithm based on Kalman filter for outlier removal and the segmentation-MI-based algorithm to further refine the correspondences and the projection matrices. The mobility of our system allows us to recover more conveniently 3-D model of static objects from the improved point cloud using a new robust radial basis function-based modeling algorithm to further suppress possible outliers and generate smooth 3-D meshes of objects. Experimental results show that the proposed 3-D reconstruction algorithm significantly reduces the adverse effect of the outliers and produces high-quality renderings using shadow light field and the model reconstructed.

Published in:

IEEE Transactions on Circuits and Systems for Video Technology  (Volume:22 ,  Issue: 10 )