Do it in OpenCV: M1: Fix a badly exposed photo, M2: Find every ball on a pool table

How Close Is Your Colour to the Real Thing?

Lesson 5 of The Image as Data. Converting between colour spaces is exact arithmetic; deciding what the numbers mean physically is a fit. Checked against a colorimeter on 22 chart patches: treating the developed file as sRGB lands 9.54 ΔE*ab from the light in the room, the camera's own published matrix does no better, and a 3x3 fitted here and scored on patches it never saw reaches 2.41.

Luis Condados · August 19, 2026 · Updated September 30, 2026 · 8 min read Intermediate

image-processing color-spaces fundamentals

Colour error against a colorimeter, by route. Source: Mark Fairchild's HDR Photographic Survey.

A photograph’s colours can be checked against a measuring instrument, and the difference is one number, ΔE, where about 1 is the smallest difference a trained eye can see. Below, each patch of the colour chart as the file records it, beside the colour a colorimeter measured.

The colour chart photograph with one patch outlined at a time; beside it two swatches, the colour in the file and the measured colour, and the error between them — Each pair is one patch: left, the file read as if it were sRGB; right, the colorimeter reading. Both are shown relative to their own white patch, as the error is computed. ΔE here is the 1976 formula, and the values are the ones this lesson reports. Source: Mark Fairchild’s HDR Photographic Survey [3].

Colour checks like this are routine where colour matters:

Paint and cosmetics matching apps compare a photo of your wall or skin with a reference.
Product photography is checked against brand colours before it goes online.
Print and display calibration measures ΔE against a target to decide when to recalibrate.

TL;DR: cvtColor is exact. What it assumes is not. Checked against a colorimeter on 22 chart patches, treating the developed file as sRGB lands 9.54 ΔE*ab from the light that was in the room. The camera’s own published characterisation does no better. A 3 × 3 fitted on this scene and scored on eleven patches it never saw reaches 2.41, and the gap between those numbers is the difference between choosing axes and knowing what they mean.

Where we are

Part of The Image as Data. Lesson 4 laid out the coordinate systems and converted a real patch through all of them. Every one of those conversions is exact arithmetic. This lesson asks the question that arithmetic cannot answer: do the numbers describe the light that was actually there?

The survey this unit measures on [3] makes that checkable: every metered patch carries a luminance and a CIE 1931 chromaticity, and luminance plus chromaticity is a full tristimulus value under the standard observer [2].

X = \frac{x Y}{y}, \qquad Y = Y, \qquad Z = \frac{(1 - x - y) Y}{y}

One decision has to be stated before any number means anything: CIELAB needs a reference white, and this scene is not lit by D65. The chart’s neutral row meters at $x = 0.464$ , $y = 0.405$ : tungsten, nowhere near D65’s (0.3127, 0.3290). So Lab is computed against the chart’s own white patch, on both sides of the comparison. Getting this wrong would report the lamp as a camera error.

Worked example · patch 15, the chart's red, from colorimeter reading to Lab

The colorimeter read patch 15 at $Y = 86.4$ cd/m², $x = 0.613$ , $y = 0.340$ . Into XYZ:

X = \frac{0.613 \times 86.4}{0.340} = 155.8, \qquad Z = \frac{(1 - 0.613 - 0.340) \times 86.4}{0.340} = 11.9.

The white is patch 19, metered at $Y = 382$ , $x = 0.464$ , $y = 0.405$ , which gives $(X_n, Y_n, Z_n) = (437.6,\ 382.0,\ 123.6)$ . Divide by the white and take cube roots (every ratio is above $\delta^3 = 0.0089$ , so the cube-root branch of $f$ applies):

\frac{X}{X_n} = 0.3559 \to 0.7087, \qquad \frac{Y}{Y_n} = 0.2262 \to 0.6093, \qquad \frac{Z}{Z_n} = 0.0967 \to 0.4589

L^* = 116(0.6093) - 16 = 54.68, \quad a^* = 500(0.7087 - 0.6093) = 49.70, \quad b^* = 200(0.6093 - 0.4589) = 30.08.

That is the reference for patch 15. The naive route reads the file’s (149.75, 58.25, 33.98) as sRGB, uses the file’s own patch 19 as white, and gets $(51.01,\ 49.05,\ 35.68)$ . The distance between the two:

\Delta E^*_{ab} = \sqrt{3.67^2 + 0.66^2 + 5.61^2} = \mathbf{6.74},

which is the artifact’s 6.73 before the displayed values were rounded.

What it means: patch 15 is off by 6.7 ΔE, a little better than the 9.54 mean. The naive route gets the red-green axis almost right ( $a^*$ differs by 0.66) and misses on lightness (3.7 too dark) and on the yellow-blue axis (5.6 too yellow). A tone curve and a white-balance mismatch are the kind of processing that errs on those two axes, which is where the rest of this lesson looks.

Three routes from photosites to Lab, over the 22 patches that do not clip:

route	what it knows about the camera	free parameters	ΔE*ab mean
treat the developed file as sRGB	nothing	0	9.54
Fairchild’s published D2x matrix	a characterisation of this camera	1 (scale)	9.14 – 10.29
a 3 × 3 fitted here, scored on held-out patches	this scene	9	2.41

What it means, one row at a time. The naive route is what most code does, and 9.54 ΔE is a long way off; for reference, 1.0 is roughly a just-noticeable difference. Its error is not spread evenly either. Down the neutral row it climbs from 0 at white to a peak of 13.20 at patch 23, and stays near that at black:

patch	19 (white)	20	21	22	23	24 (black)
ΔE*ab	0.00	1.18	4.45	8.27	13.20	12.96

That is unit 1.2’s response curve seen from the colour side: the developed file is not proportional to light, and the discrepancy piles up in the shadows.

The middle row is the surprise, and it needs care. Fairchild’s own characterisation of this camera, a 3 × 3 he fitted to this very scene’s lit ColorChecker and published [4], does not beat the naive route here. It was applied four ways: his scene-fitted matrix and his D65-normalised one, each on raw and on white-balanced linear RGB. All four land between 9.14 and 10.29. One configuration going wrong would suggest a mistake on our side; four landing in the same place says the problem sits upstream of the choice between them.

The lamp cannot be the reason, because the matrix was fitted under this same lamp. What differs is the RGB it was fitted to. Fairchild’s fit used HDR images merged from Photoshop CS2 raw conversions, white-balanced with the camera’s own white point [3]; here the matrix is applied to the linear raw mosaic, which has been through none of that. I believe that mismatch is responsible. I have not reproduced his conversion, so the belief is untested.

The last row is the only real prediction of the three. A 3 × 3 fitted on 11 patches and scored on the 11 it never saw reaches 2.41 ΔE mean, 2.01 median, four times better than the naive route, on patches that had no say in the fit. For comparison, Fairchild reports 2.5 mean for his matrix on the 25 patches it was fitted to (the lit checker’s 24 and the black behind it), which is a residual rather than a prediction; ours is 1.49 on its training half, so the two numbers describe the same quality of fit.

The reading that matters: a large naive ΔE is a statement about the rendering chain (the tone curve, the white balance, the assumption of sRGB primaries) and says nothing against cvtColor’s arithmetic, which is exact. The tone curve is unit 1.2’s response curve, and the interpolated two thirds of every colour pixel come from the Bayer mosaic. Choosing axes is free. Knowing what the axes mean in physical terms costs a calibration.

Only the bottom route is a prediction: it was fitted on eleven patches and scored on the eleven it never saw. The middle one is a published fit to this same scene, applied to linear raw RGB rather than the converted RGB it was fitted to.

Now you try

Three channel gains and a tone exponent, against the 22 metered patches. Start from the naive route and try to bring the mean down.

You can reach about 4 with patience, which is most of the way, and then it stops. What is left needs the off-diagonal terms of a full 3 × 3, because the camera’s red channel sees some of what the eye calls green, and no amount of per-channel scaling expresses that.

In the wild

The lesson above needed a colorimeter. Here is a version of the same check that needs nothing but the subject, and you can run it on any photograph that happens to contain a colour chart.

A ColorChecker’s bottom row is neutral by manufacture. Whatever a camera reports for those six patches, the true answer for both colour axes is zero. So the departure from zero is the error, with no reference table to look up and nothing to take on trust.

Measured on a photograph of a chart, converting the file to Lab the same way lesson 4 does:

patch	19 (white)	20	21	22	23	24 (black)
L*	88.0	81.3	69.3	50.7	29.0	4.4
a*	0.85	0.68	0.04	0.00	−0.81	−0.31
b*	−2.93	−3.30	−1.67	−0.19	0.02	0.66
distance from neutral	3.05	3.37	1.67	0.19	0.81	0.73

A photographed ColorChecker with the six patches of its neutral bottom row outlined and numbered — The row that should have no colour in it at all. Source: “ColorChecker100423” by ColorScientist (Wikimedia Commons), public domain.

What it means: a mean of 1.64 and a worst of 3.37, where the correct answer is 0.00 and where 1.0 is roughly the smallest difference an eye can see. And the shape of the error is the informative part: it is the light patches that are off, both of them towards blue, while the dark ones sit close to neutral. A cast that grows with brightness looks like a white-balance error rather than noise: negative $b^*$ is blue, so the photograph was probably balanced for a warmer light than the one that was there, and the correction added too much blue. One photograph cannot confirm which setting was used.

This is the poor-relation version of the colorimeter comparison, and it is worth knowing because it costs nothing. If a chart is in the frame, your pipeline can check its own colour with no ground truth beyond the fact that grey is grey.

Where this breaks

The fitted matrix is fitted to this scene and this lamp, and I would not expect it to transfer to daylight. Nine parameters on eleven training patches is also a generous ratio; the held-out score guards against memorising individual patches, but not against a systematic bias shared by all of them.

The mechanism behind the middle row is untested. Checking it needs Fairchild’s Photoshop CS2 conversion reproduced and his matrix applied to that output instead of the raw mosaic.

ΔE*ab is the 1976 formula, chosen because it is what Fairchild’s characterisation reports and the comparison has to be like for like. It is known to overstate differences in saturated colours; CIEDE2000 would give smaller numbers for the same errors, and the ordering of the three routes would not change.

Finally, the patches are flat interiors under one lamp, which is the friendliest test available. Real scenes have mixed illuminants, and a single white point cannot serve two of them at once.

File formats: what the encoder did: the last thing that happens to the array before you read it, and the first thing that happened to the one you loaded.

References

[1] Reinhard, E., Khan, E. A., Akyüz, A. O., & Johnson, G. M. (2008). Color Imaging: Fundamentals and Applications. A K Peters/CRC Press.

[2] International Commission on Illumination (2004). CIE 15: Technical Report: Colorimetry (3rd ed.). The CIE 1931 observer and the CIELAB definition used here. CIE.

[3] Fairchild, M. D. (2007). The HDR Photographic Survey. Proceedings of the IS&T 15th Color and Imaging Conference, pp. 233–238. doi:10.2352/CIC.2007.15.1.art00044. The scene this unit measures. Used for research and non-commercial publication, as its terms require; images are downloaded, never redistributed.

[4] Fairchild, M. D. (2007). Nikon D2x Characterization for HDR-from-NEF Images (updated 21 May 2007). Rochester Institute of Technology. Characterisation notes. The published RGB→XYZ matrices and their ΔE*ab residuals.

Tagged