Optimal radiometric calibration for camera-display communication

We present a novel method for communicating between a moving camera and an electronic display by embedding and recovering hidden, dynamic information within an image. A small intensity pattern is added to alternate frames of a time-varying display. A handheld camera pointed at the display can receive not only the display image, but also an underlying message. Differencing the camera-captured alternate frames leaves the small intensity pattern, but results in errors due to photometric effects that depend on camera pose. Detecting and robustly decoding the message requires careful photometric modeling for message recovery. The key innovation of our approach is an algorithm that performs simultaneous radiometric calibration and message recovery in one convex optimization problem. By modeling the photometry of the system using a camera-display transfer function (CDTF), we derive an optimal online radiometric calibration (OORC) for robust computational messaging as demonstrated with nine different commercial cameras and displays. The online radiometric calibration algorithms described in this paper significantly reduces message recovery errors, especially for low intensity messages and oblique camera angles.