Results per Dataset (Reproduction error)

This page provides the Reproduction Angular Errors of several illuminant estimation algorithms for different datasets. Reproduction Angular Error was proposed by Graham D. Finlayson and Roshanak Zakizadeh in British Machine Vision Conference (BMVC) 2014. The paper can be downloaded from here.

Data sets:

SFU Laboratory Images

Note that illuminant estimates are available for a wide range of parameter-settings (i.e. Minkowski-norm is varied between 1 and 15, and the smoothing value σ is varied between 1 and 12).

Method (applied to 321 images)  Mean error   Median error   Trimean error   95% quantile error   
Grey-World: 10.1o 7.5o 8.3o 28o
White-Patch: 9.7o 7.4o 8.2o 27.3o
Shades-of-Grey: (p = 7) 6.9o 3.9o 4.8o 22.5o
general Grey-World (p = 10, σ = 4) 6.0o 3.9o 4.3o 20.2o
1st-order Grey-Edge (p = 14, σ = 4) 6.3o 3.6o 4.2o 21.8o
2nd-order Grey-Edge (p = 15, σ = 10) 5.8o 3.0o 3.8o 24.7o
Pixel-based Gamut (σ = 4) 4.2o 2.8o 3.0o 14.8o
Edge-based Gamut (σ = 2) 4.5o 2.7o 3.2o 14.3o
Union-based Gamut (σ = 2) 5.2o 3.4o 3.9o 15.3o
Intersection-based Gamut (σ = 3) 4.2o 2.5o 2.8o 17.6o
Inv. Intensity Chrom. Space 15.1o 9.3o 11.5o 43.2o
Spatial Correlations:
 — HeavyTailed-based
 
6.2o
 
4.1o
 
5.0o
 
16.6o
 
Weighted Grey-Edge:
 — p = 2, σ = 1, κ = 8
 
6.1o
 
3.6o
 
4.3o
 
19.3o
 

SFU Grey-ball Set (Original)

The original Grey-ball set consists of 11,346 images. However, because of the large correlation among some consecutive frames, various researchers decided to use a subset of this set. Feel free to use the following data to extract any results appropriate, since they are computed by processing the full set:

Method (applied to 11346 images)  Mean error   Median error   Trimean error   95% quantile error   
Grey-World 8.7o 7.6o 7.9o 20.9o
White-Patch 7.1o 5.5o 6.0o 18.0o
Shades-of-Grey (p = 14) 6.5o 5.6o 5.8o 15.2o
general Grey-World (p = 12, σ = 0) 7.1o 6.2o 6.4o 16.6o
1st-order Grey-Edge (p = 2, σ = 1) 6.3o 4.8o 5.4o 15.7o
2nd-order Grey-Edge (p = 1, σ = 2) 6.5o 5.0o 5.6o 15.7o
Pixel-based Gamut (σ = 2) 7.5o 5.9o 6.3o 19.6o
Edge-based Gamut (σ = 1) 7.3o 5.8o 6.3o 18.1o
Intersection-based Gamut (σ = 2) 7.5o 5.9o 6.4o 19.6o
Inv. Intensity Chrom. Space 7.0o 6.0o 6.2o 15.9o
Using Natural Image Statistics 5.5o 4.3o 4.7o 13.7o
Exemplar-Based Color Constancy 4.8o 3.7o 4.0o 12.5o
Color Cat 4.6o 3.4o 3.8o 12.6o
ColorDog:
 — Using WP / GW
 
4.9o
 
3.2o
 
3.8o
 
13.4o
 — Using ColorCat 5.7o 4.3o 4.7o 15.8o

Color-checker (by Shi)

Since the original color checker set wat generated from RAW data using automatic settings, Shi reprocessed the original RAW data and generated 12-bit PNG-images (with lossless compression).

Method (applied to 568 images)  Mean error   Median error   Trimean error   95% quantile error   
Grey-World 7.0o 6.8o 6.9o 12.4o
White-Patch 8.1o 6.5o 7.1o 20.0o
Shades-of-Grey (p = 5) 5.8o 4.4o 4.9o 14.7o
general Grey-World (p = 9, σ = 9) 5.3o 4.0o 4.4o 14.0o
1st-order Grey-Edge (p = 1, σ = 9) 6.4o 4.9o 5.3o 15.8o
2nd-order Grey-Edge (p = 1, σ = 1) 6.0o 4.8o 5.2o 13.7o
Pixel-based Gamut (σ = 4) 4.8o 2.7o 3.4o 15.8o
Edge-based Gamut (σ = 3) 8.0o 5.9o 6.6o 20.5o
Intersection-based Gamut (σ = 4) 4.8o 2.7o 3.4o 16.4o
Inv. Intensity Chrom. Space 14.3o 13.6o 13.6o 22.6o
Regression (SVR) 8.8o 7.4o 7.9o 18.9o
Bayesian 5.6o 3.9o 4.4o 15.4o
Spatial Correlations:
 — Gaussian-based
 
o
 
o
 
o
 
o
 — HeavyTailed-based 4.2o 3.5o 3.6o 9.9o
Bottom-up (see here) 4.0o 3.0o 3.2o 11.6o
Top-down (see here) 4.3o 3.1o 3.2o 13.8o
Bottom-up+Top-down (see here) 4.0o 2.8o 3.0o 13.6o
Using Natural Image Statistics 4.8o 3.5o 3.9o 12.9o
CART-based Selection 5.2o 4.0o 4.1o 14.6o
CART-based Combination 4.5o 3.5o 3.8o 11.4o
Exemplar-Based Color Constancy 3.4o 2.6o 2.9o 8.3o
Using CNNs:
 — AlexNet+SVR
 
5.5o
 
3.6o
 
4.1o
 
16.3o
 — Deep color constancy 3.1o 2.2o 2.5o 9.0o