IEEE Transactions on Circuits and Systems for Video Technology

Offline or deferred solutions are frequently employed for high quality and reliable results in current video stabilization. However, neither of these solutions can be used for strict real-time applications. In this paper, we propose a practical and robust algorithm for real-time video stabilization. To achieve this, a novel and efficient motion model based on inter-frame homography estimation is p... View full abstract»

This paper investigates how to fuse grayscale and thermal video data for detecting foreground objects in challenging scenarios. To this end, we propose an intuitive yet effective method called weighted low-rank decomposition (WELD), which adaptively pursues the cross-modality low-rank representation. Specifically, we form two data matrices by accumulating sequential frames from the grayscale and t... View full abstract»

In this paper, we propose a novel method for 4D light-field (LF) depth estimation exploiting the special linear structure of an epipolar plane image (EPI) and locally linear embedding (LLE). Without high computational complexity, depth maps are locally estimated by locating the optimal slope of each line segmentation on the EPIs, which are projected by the corresponding scene points. For each pixe... View full abstract»

In this paper, we propose a novel method for automatic depth estimation from color images using parameter transfer. By modeling the correlation between color images and their depth maps with a set of parameters, we get a database of parameter sets. Given an input image, we extract the high-level features to find the best matched image sets from the database. Then the set of parameters correspondin... View full abstract»

A human body scanning system of great practical convenience, which can be used in an outdoor environment, is proposed. The system uses only a single conventional video camera without the aid of special sensors or controlled illuminations. We leverage the structure from motion calibration results directly and improve the available video-based dense 3D reconstruction by integrating the surface smoot... View full abstract»

Photorealistic animation is a desirable technique for computer games and movie production. We propose a new method to synthesize plausible videos of human actors with new motions using a single cheap RGB-D camera. A small database is captured in a usual office environment, which happens only once for synthesizing different motions. We propose a marker-less performance capture method using sparse d... View full abstract»

As of today, fine-grained editing of facial performances in movie and video production requires either retouching every single frame or creating a highly detailed CGI model of the actor, both of which is restricted to high-budget productions. In this paper, we present an example-based approach for facial performance editing that achieves realistic results with standard equipment and very little ma... View full abstract»

The latest developments in 3D capturing, processing, and rendering provide means to unlock novel 3D application pathways. The main elements of an integrated platform, which target tele-immersion and future 3D applications, are described in this paper, addressing the tasks of real-time capturing, robust 3D human shape/appearance reconstruction, and skeleton-based motion tracking. More specifically,... View full abstract»

This paper presents a mixed reality (MR) system that results from the integration of a telepresence system and an application to improve collaborative space exploration. The system combines free viewpoint video with immersive projection technology to support nonverbal communication (NVC), including eye gaze, interpersonal distance, and facial expression. Importantly, these features can be interpre... View full abstract»

We present a generic and real-time time-varying point cloud codec for 3D immersive video. This codec is suitable for mixed reality applications in which 3D point clouds are acquired at a fast rate. In this codec, intra frames are coded progressively in an octree subdivision. To further exploit interframe dependencies, we present an inter-prediction algorithm that partitions the octree voxel space ... View full abstract»

This paper proposes a virtual 3D eyeglasses try-on system driven by a 2D Internet image of a human face wearing with a pair of eyeglasses. The main technical challenge of this system is the automatic 3D eyeglasses model reconstruction from the 2D glasses on a frontal human face. Against this challenge, this paper first proposes an eyeglasses segmentation method using a convolutional neural network... View full abstract»

Light field (LF) acquisition faces the challenge of extremely bulky data. Available hardware solutions usually compromise the sensor resource between spatial and angular resolutions. In this paper, a compressed sensing framework is proposed for the sampling and reconstruction of a high-resolution LF based on a coded aperture camera. First, an LF dictionary based on perspective shifting is proposed... View full abstract»

Online object tracking is a challenging issue because the appearance of an object tends to change due to intrinsic or extrinsic factors. In this paper, we propose a tracking algorithm based on probabilistic hypergraph ranking. First, three types of hypergraphs are constructed to encode local affinity information. Then, a probabilistic hypergraph is built by combining three distinct hypergraphs lin... View full abstract»

A novel visual object tracking method based on consistency-constrained nonnegative coding (CNC) is proposed in this paper. For the purpose of computational efficiency, superpixels are first extracted from each observed video frame. And then CNC is performed based on those obtained superpixels, where the locality on manifold is preserved by enforcing the temporal and spatial smoothness. The coding ... View full abstract»

This paper proposes an extended selective encryption (SE) method for both H.264/advanced video coding (AVC) (CABAC) and High Efficiency Video Coding (HEVC) streams, addressing the main security issue that SE is facing: content protection, related to the amount of information leakage through a protected video. Our contribution is the improvement in the visual distortion induced by SE approaches. Pr... View full abstract»

Digital video stabilization is an important video enhancement technology that aims to remove unwanted camera vibrations from video sequences. Trading off between stabilization performance and real-time hardware implementation feasibility, this paper presents a feature-based full-frame video stabilization method and a novel complete fully pipelined architectural design to implement it on field-prog... View full abstract»

This paper addresses automatic video summarization. We propose a novel approach that relies on constraint satisfaction programming (CSP). An expert defines the general rules for summary generation. These rules are written as constraints. The (final) user can define additional constraints or enter high-level parameters of predefined constraints. This has many advantages. It clearly separates summar... View full abstract»