Consistent real-time lighting for virtual objects in augmented reality

We present a technique for rendering realistic shadows of virtual objects in a mixed reality environment by recovering the light source distribution of a scene in real-time, through the segmentation and analysis of a known occluding object's shadows. A fiducial marker provides information about the position of the occluding object and the plane of the surface on which shadows are cast, and serves as the origin of a marker coordinate system. A new shadow segmentation approach is carried out on the shadow image and is able to recover geometrical information on multiple faint shadows. Using normalised iterative reinforcement, noise and artifacts can be suppressed in the final shadow map. The scene's light source distribution is then extrapolated using geometrical data from both the occluding object and its cast shadows. Virtual light sources in a game engine are used to mimic real light sources and achieve consistent illumination and increase the realism of the augmented reality scene.