The Image as Data: The Grid, the Integers, the Axes, and the File

An image is a grid of integers, and every choice made in building that grid has a cost you can measure.

That is the unit. Five lessons: what the array actually holds, where its samples sit in the world, how finely each sample is written down, what its three numbers mean, and what the file did to all four before you ever called imread.

Before this

You need image sensing — what a raw count is, the gain that produced it, the noise it carries, and the tone curve the camera applied on the way out. This unit spends all four of those and re-derives none of them. The clearest case is lesson 3, where the question “how many bits does this camera deserve?” is answered entirely with a noise figure that unit measured.

The scene, and the three files it comes in

Every number here comes from one frame — _MDF0005.NEF, the 1/45 s exposure of “Luxo Double Checker” from Mark Fairchild’s HDR Photographic Survey [3], which is the same scene image sensing measured. A desk lamp with the bulb lit, a GretagMacbeth ColorChecker under it, and a ribbed table running out of the frame.

What makes it the right scene for this unit is not the lamp. It is that one photograph can be held three ways at once:

	shape	dtype	in memory	what it is
raw mosaic	2868 × 4312	uint16, 12 bits used	24.7 MB	one number per photosite — the only measurement
developed, 16-bit	2868 × 4312 × 3	uint16	74.2 MB	three numbers per pixel, two of them invented
developed, 8-bit	2868 × 4312 × 3	uint8	37.1 MB	what imread would hand you

All three sit on one pixel grid, because all three are developed here from that single NEF. (The survey publishes a rendered JPEG too, and it is 600 × 337 — a thumbnail. Its OpenEXR is a merge on a different grid again. Neither can be compared pixel for pixel with the raw, which is why this unit builds its own.)

The chart is the other half of the scenario. Fairchild metered its patches with a Konica Minolta CS-100 and published a luminance and a CIE 1931 chromaticity for each — so lesson 4 can check a colour conversion against an instrument that never saw our code.

And the same ideas, on photographs

The scene above measures well because it was built to. It is also a bench setup in a dark room, and nothing you photograph looks like it — so every lesson repeats its measurement on a real picture and puts the two numbers side by side. The lab frame says what an effect is; the photograph says what it is worth.

lesson	in the wild	what it adds
1	a red telephone box	the channel swap, on a subject nobody can mistake: 99.7% of pixels differ
2	a brick wall, and a spinning wheel	a thumbnail at 81 px falls below Nyquist; and a wheel filmed at 24 fps that runs backwards at −0.90 rev/s
3	twilight over La Silla	banding, which the lab scene has no gradient to show: 51 bands become 3
4	a Marrakech fruit stall	a hue window that cannot separate oranges from lemons, because they are neighbours in hue
5	a photographed ColorChecker	grey that should be grey, off by 1.64 — a colour check needing no instrument
6	an 1814 newspaper	text survives q25 at 0.988 edge F1 while the paper beside it takes 2.3× the error

Every one of those is a Wikimedia Commons file under CC0, public domain or CC BY, with the licence verified and the bytes pinned by sha256 in CondadosAI/cv-assets. They are downloaded, never redistributed; the posts publish derived figures with the credit in the caption.

Topics

1. An image is an array — what does imread actually hand you, and what do its shape, dtype and channel order cost you?
2. Sampling: what one pixel covers — where do the samples sit in the world, and what happens to whatever falls between them?
3. Quantization: how many bits a pixel deserves — how finely should each sample be written down, and when does finer stop helping?
4. Colour spaces: choosing axes you can threshold — what coordinate systems can those three numbers be written in, and why move between them?
5. How close is your colour to the real thing? — the conversions are exact, but do the numbers describe the light that was there?
6. File formats: what the encoder did — what did the file throw away, and does anything downstream notice?

Where the three colour numbers come from is not here: a sensor measures one of them per photosite and interpolates the rest, which is unit 1.2’s last lesson.

How they fit together

What the unit measures

Six headline numbers, one per lesson, each derived in the post that owns it:

lesson	the number
1 — the array	two thirds of every developed colour image is interpolated, not measured
2 — sampling	a 7.4-px ribbing, decimated by 8, returns at 96 px — 13× too coarse
3 — quantization	quantization error meets this sensor’s noise at 9.3–9.5 bits, so a 12-bit file carries ~2.5 spare
4 — colour spaces	one patch, RGB (149.8, 58.3, 34.0), is luma 82.8 and a hue of 12° that OpenCV stores as 6
5 — colour accuracy	the naive route from file to Lab lands 9.54 ΔE*ab from the metered colour; a fitted 3 × 3 reaches 2.41 on held-out patches
6 — file formats	q95 → q25 costs under 4 dB of PSNR and takes edge agreement from 0.949 to 0.895

Scope, and what these numbers are not

This hub carries a table, so it owes the same honest scope section as any lesson.

Every number above is one camera, one scene, one frame — a Nikon D2X at ISO 100, in a dark room lit by a single tungsten lamp. The compression ratios in particular are a fact about that darkness as much as about the encoders: nine tenths of the frame is near-black, and lesson 5 reports the content crop separately for exactly that reason. The 9.4-bit crossing is this sensor at this ISO. The 7.4-pixel ribbing is one window of one table, and the period drifts from 6.2 to 13.7 px across the crop.

No model was run anywhere in this unit. Nothing here is a claim about detection accuracy, and lesson 6 says so in those words where the temptation is strongest.

Where this lands

No post on the site yet measures what sampling, quantization or compression cost a model on real hardware, so this unit does not claim one. It ships as a fundamentals unit and the closer is pending. If you want to see where these arrays end up, INT8 instance segmentation on an Intel iGPU is the nearest thing — it quantizes the model rather than the pixels, which is a different subject that happens to rhyme.

Reproducibility

Data	HDR Photographic Survey, “Luxo Double Checker”, _MDF0005.NEF [3]
Camera	Nikon D2X, 12-bit NEF, 23.7 × 15.7 mm, 4288 × 2848 effective
Development	rawpy.postprocess(use_camera_wb=True, no_auto_bright=True) at 8 and 16 bits
Software	Python 3.12.9, NumPy 2.5.2, OpenCV 5.0.0, Pillow 12.3.0, rawpy 0.27.0 (LibRaw 0.22.1)
OS	Linux 6.8.0-136-generic
Runs	Deterministic throughout; no timing is reported, so no repeats
In the wild	Seven Wikimedia Commons files — CC0, public domain or CC BY 4.0, none share-alike. Registered in CondadosAI/cv-assets with the licence read from the Commons API and a sha256 pinning the bytes measured; fetched by uv run data-experiments, never redistributed
Artifact	output/image_data_numbers.json, via uv run data-experiments. Section keys l1_array … l5_formats, cfa and in_the_wild

Every number in every lesson of this unit is in that one file. The lessons themselves carry no reproducibility table — this one covers all six, which is how unit 3.1 has always worked and how the rest of the track now works too. The survey’s images and the Commons files are downloaded and never redistributed; what the posts publish is derived figures, credited in each caption.

Further reading

• Go deeper: Gonzalez & Woods, Digital Image Processing (4th ed.) [1] — §2.4 (Image Sampling and Quantization, p. 47) covers lessons 1 to 3, ch. 7 (Color Image Processing, p. 529) covers lessons 4 and 5, and §8.9 (Block Transform Coding, p. 632) covers lesson 6. One chapter map for the whole unit.
• The API: the OpenCV colour-conversion documentation [2], which is where the 8-bit hue range and the channel order are actually specified.
• Related on CondadosAI: image sensing is the prerequisite · 2D convolution is the first thing anyone does to the array once they have it.

References

[1] Gonzalez, R. C., & Woods, R. E. (2018). Digital Image Processing (4th ed.), §2.4 (Image Sampling and Quantization, p. 47), ch. 7 (Color Image Processing, p. 529) and §8.9 (Block Transform Coding, p. 632). Pearson. Detailed table of contents.

[2] OpenCV. Color conversions (cvtColor), OpenCV 5.0.0 documentation. Docs.

[3] Fairchild, M. D. (2007). The HDR Photographic Survey. Proceedings of the IS&T 15th Color and Imaging Conference, pp. 233–238. doi:10.2352/CIC.2007.15.1.art00044 — the scene this unit measures. Used for research and non-commercial publication, as its terms require; images are downloaded, never redistributed.