Detecting Illumination in Images

In this paper we present a surprisingly simple yet powerful method for detecting illumination—determining which pixels are lit by different lights—in images. Our method is based on the chromagenic camera, which takes two pictures of each scene: one is captured as normal and the other through a coloured filter. Previous research has shown that the relationship between the colours, the RGBs, in the filtered and unfiltered images depends strongly on the colour of the light and this can be used to estimate the colour of the illuminant. While chromagenic illuminant estimation often works well it can and does fail and so is not itself a direct solution to the illuminant detection problem. In this paper we dispense with the goal of illumination estimation and seek only to use the chromagenic effect to find out which parts of a scene are illuminated by the same lights. The simplest implementation of our idea involves a combinatorial search. We precompute a dictionary of possible illuminant relations—that might map RGBs to filtered counterparts—from which we select a small number 𝑚 corresponding to the number of distinct lights we think might be present. Each pixel, or region, is assigned the relation from this 𝑚-set that best maps filtered to unfiltered RGB. All 𝑚-sets are tried in turn and the one that has the minimum prediction error over all is found. At the end of this search process each pixel or region is assigned an integer between 1 and 𝑚 indicating which of the m lights are thought to have illuminated the region. Our simple search algorithm is possible when 𝑚 = 2 (and 𝑚 = 3) and for this case we present experiments that show our method does a remarkable job in detecting illumination in images: if the 2 lights are shadow and non- shadow, we find the shadows almost effortlessly. Compared to ground truth data, our method delivers close to optimal performance.