Augmented TV: An augmented reality system for TV programs beyond the TV screen

TV Service using a mobile device as a second screen has been increasing. We propose a new TV system which is able to augment representation of TV programs beyond the TV screen. In the system, which we named augmented TV, since animated 3DCG content interlocked with TV programs is overlaid on live video from the mobile device camera in the mobile device screen by augmented reality techniques, the representation of having a TV character coming out of the screen can be provided. To achieve the representation giving such surprise or reality to the viewer, synchronization accuracy of the overlay display is required. A conventional synchronization method for multi-device or a visible light communication method does not meet the requirement of the accuracy. Therefore, we developed an accurate synchronization method and authoring environment of augmented TV content. We implemented augmented TV, and confirmed frame-accurate synchronization (synchronization error time is about 0.03 seconds or less). And we confirmed that the authoring environment is easy to produce augmented TV content using a video clip using a 3DCG character with TV program quality.