What is color constancy?

Color Constancy is the ability to perceive colors of objects, invariant to the color of the light source. This ability is generally accredited to the Human Visual System, although the exact details remain uncertain. An example of the ability is shown in the figure below.

Illustration of Color Constancy

In this example, the same flower is depicted four times, each rendered under a different light source. As can be seen, the color of the flower is strongly dependent on the color of the light source. Computational Color Constancy can follow different paths to maintain stable color appearance across light sources. One common path, which is now believed not to mimic the human visual system, but is very common among computational models, approaches the problem using two phases. First, based on several assumptions, the color of the light source is estimated from an input image.  Then, using this estimated illuminant, the input image is corrected so that it appears to be taken under a canonical (e.g. white) light source. This approach is outlined in the figure below.

Illustration of Color Constancy through Illuminant Estimation

The original image is recorded under a blueish light source. Assuming uniform illumination across the image, the estimated light source (the blue pane) is used to correct every pixel in the input image. The output image is the ultimate goal, but the main focus of this website is on the first phase: illuminant estimation.

Image Formation

One common assumption in literature on color constancy is Lambertian shading, i.e. assuming an image consists of only matte, dull surfaces. In this case, an image f = (fR, fG, fB) is composed of the multiplication of three terms, i.e. the color of the light source I(x,\lambda), the surface reflectance properties S(x,\lambda) and the camera sensitivity function c(\lambda): where c = {R,G,B}, over the the visible spectrum \omega, m(x) is the Lambertian shading, \lambda is the wavelength of the light and x is the spatial coordinate in the image. Further assumptions include a spectrally uniform light source, i.e. I(x,\lambda) = I(x) for all locations x in the image. Then, the observed color of the light source e depends on the spectrum of the light and the camera sensitivity function: Since there are two unknown variables (the surface reflectance function S(x,\lambda) and the color of the light source e) and only one known variable (the image values f), the estimation of e is an under-constrained problem. A common approach to solving this problem is to make further assumptions, for instance on the distribution of image colors or on the set of possible light sources. These assumptions will further be explained here.

Image Correction

Once the chromaticity of the light source is known, the input image can be corrected. The transformation to convert an input image, recorded under an unknown light source, to an output image that appears to be recorded under a canonical light source, is called chromatic adaptation. Chromatic adaption is often modeled using a linear transformation, which in turn can be simplified by a diagonal transformation when certain conditions are met. On this website, the diagonal model is used to correct the input images. Alternatives to this include CIECAT02 (part of CIECAM02) and linearized Bradford. The diagonal model is given by: where fu is the image taken under an unknown light source, ft is the same image transformed, so that it appears if it was taken under a canonical light source, and Mu,t is a diagonal matrix which maps colors that are taken under an unknown light source u to their corresponding colors under the canonical illuminant t.