Simultaneous estimation of illuminant and object colors

Purpose From perceived colors in everyday scenes, observers routinely infer properties of the illuminant such as sunny versus cloudy, or summer sun versus winter sun. This requires inferring object invariants and illumination caused color changes. We present methods to simultaneously approximate illuminant and object chromaticities by using photoreceptor or opponent-color signals from views of a scene under a pair of illuminants. Methods The local coordinate frame analysis used to infer three-dimensional shape from two views by Koenderink and van Doom (1991), was applied to (L,M,S) and (L+M, L-M, L+M-S) signals calculated for collections of natural and man-made objects under different illuminants. Results Spectral changes in illumination lead to linear transformations of sets of object chromaticities. In principle, a simple algorithm operating on non-coplanar signals from four objects under a pair of illuminants, is sufficient to infer object chromaticities in a three-dimensional affine color space, along with the direction of the chromaticity shift between the views. Formally, a singular value decomposition achieves the same purpose. Weaker conditions suffice for recovering object colors than are required for object shapes, because recovering metrical relationships is not an issue for color spaces. Conclusions Photoreceptor signals from a variegated scene carry information about objects and illuminants. The two types of information are useful in different tasks, therefore discounting either at an early stage is suboptimal. The linear structure imposed by illuminant shifts makes it possible to simultaneously estimate both types of information.