Sampling: What One Pixel Covers, and What Falls Between

A digital image only knows the scene at the points where its pixels sit, so detail finer than the spacing between pixels turns into patterns that are not there. Below, a patch of a net made smaller by keeping every k-th pixel, beside the same patch blurred first.

You have seen aliasing:

• Moiré on a photo of a screen, a striped shirt or a fence.
• Shimmering thumbnails when an image is shrunk without smoothing first.
• Wheels turning backwards in video, the same folding in time rather than space.

TL;DR: one photosite on this camera is 5.527 µm across and covers 1.056 arcminutes of the room. The table in the frame is ribbed at 7.4 pixels per cycle. Keep every 8th pixel and that ribbing comes back at 96 pixels per cycle, thirteen times too coarse, and a pattern the room does not contain. Where it lands is arithmetic, and the measurement agrees with the arithmetic in every case.

Where we are

Part of The Image as Data. Lesson 1 left the array knowing what its numbers are and nothing about where they came from. Where those photosites physically sit, and why the optics in front of them matter, is unit 1.1’s pinhole and perspective projection. This lesson is about the grid: how far apart the samples are, and what happens to whatever lies between them.

What one sample covers

A photosite is a small square that collects everything falling on it, so a pixel is an average over a patch of the world rather than a point measurement.

The D2X puts 4288 effective pixels across a 23.7 mm sensor [2], so

and the vertical pitch works out to 5.513 µm from the other side of the sensor. They agree to 0.014 µm, which is the check that the sensor is on a square grid. At this scene’s 18 mm focal length, one photosite subtends

What it means: roughly one arcminute per pixel, which is about the resolving limit of a good human eye. The chart’s white patch is sampled 2,916 times inside its measurement box alone. Its border is sampled once. Every error a grid makes lives at the edges, because that is the only place where one sample has to stand for two things.

The pattern that is finer than the grid

The table under the lamp is ribbed, and that ribbing is the only strongly periodic thing in the frame, which makes it the one place where sampling can be measured rather than asserted.

Measuring it turned up something the plan did not expect. The ribbing has no single period. Measured over a 192-row window that starts at each frame row below and slides down in 96-row steps, it runs from 6.19 px per cycle at the far edge of the crop to 13.71 px at the near edge, growing at every step:

That drift is perspective: a physical grating photographed at a shallow angle is foreshortened at the far end. So the frequency work below runs on one of those windows, frame rows 2196 to 2388 (the second column), where the FFT peak sits at 26 cycles in 192 samples: a period of $192/26 =$ 7.385 px, the table’s 7.38 to one more decimal. The period still drifts inside the window (the next window, which overlaps half of it, reads 8.00), so 7.385 is that window’s dominant period, and every prediction below is scored against the FFT of the same window.

Where an undersampled pattern goes

If you only look at every $k$-th pixel, a fast wiggle and a slow wiggle can produce the identical sequence of numbers, and nothing downstream can tell them apart.

Keeping every $k$-th sample multiplies the frequency in the new grid by $k$. Anything past half a cycle per sample has nowhere to go and reflects back:

The ×4 row matters as much as the ×8 one: 1.85 samples per cycle is above one and still below two, and it already aliases. In all three cases the measured peak falls in the bin the fold predicts. The bin is 0.010, 0.021 and 0.042 cycles per new pixel respectively, which is coarse at ×8, so the claim is “lands in the predicted bin” and no finer than that.

What this costs when you resize

The fix is to average over the footprint before sampling, which is what cv2.INTER_AREA does [4]. (Averaging before sampling is a filter, and what a filter does to an image is the subject of 2D convolution.) Measured against that reference on the ribbed crop:

What it means: bilinear and area agree to the last digit at ×2, then diverge. INTER_LINEAR is cv2.resize’s default and it reads only a 2 × 2 neighbourhood, so past ×2 it is skipping most of the pixels it is supposed to be averaging. On the flat white patch, where there is nothing above Nyquist to lose, the same ×4 comparison gives 1.874 and 0.963, and the methods barely differ. Aliasing is a property of the content more than of the resize call.

The same resize happens silently inside most model pipelines. The PatchCore vs VLM post on VisA feeds photos 1,274 to 1,562 pixels wide into a 256-pixel input, and moving that input to 512 pixels adds 3.7 to 6.7 points of AUROC at every number of good parts it tested. Warping an image resamples it too, and image warping and blending is where the interpolation choice meets a homography.

small = cv2.resize(img, (w // 8, h // 8), interpolation=cv2.INTER_AREA)   # averages
naive = cv2.resize(img, (w // 8, h // 8), interpolation=cv2.INTER_NEAREST)  # samples
# INTER_LINEAR is the default and reads a 2x2 neighbourhood: fine at x2, not at x8.

cv::Mat small, naive;
cv::resize(img, small, cv::Size(w / 8, h / 8), 0, 0, cv::INTER_AREA);
cv::resize(img, naive, cv::Size(w / 8, h / 8), 0, 0, cv::INTER_NEAREST);

Now you try

Leave the period at the measured 7.385 px and walk the sampling interval up from 1. Nothing happens until you pass 3; then the reconstruction detaches from the signal completely. Now nudge the period to 7.4, a change of 0.015 px, and the answer at ×8 moves from 96 px to 99. Folding multiplies any error in the period by the sampling interval, which is why the article names the window its 7.385 was measured in.

In the wild

Two of them, because this is the one lesson where a still picture cannot show the whole idea.

First, a brick wall, and the moment you make a thumbnail. Photographed square-on at 2592 × 1944, the courses of brick repeat every 54.0 pixels. That is enormous, so shrinking to 648 px or even 162 px is perfectly safe: 13.5 and 3.4 samples per cycle, both above the two that Nyquist asks for. Then:

What it means: 81 pixels wide is a thumbnail and 40 pixels is an avatar. Those are the sizes of a gallery grid and a comment avatar, and they are where a regular texture stops being itself. At ÷64 the wall comes back as a pattern six times too coarse, which is why brickwork and pinstripes look so strange in small previews.

Second, the version you cannot print. A grid samples space; a camera also samples time, and everything above applies to frames the same way it applies to pixels. Filmed at 24 frames per second, a spoked wheel turning forwards can be recorded turning backwards.

What it means: the sign is the finding. The wheel is neither ambiguous nor blurred. It is confidently rotating the wrong way, and no processing of those frames can recover the truth, because the frames are consistent with the wrong answer. One limit: a spoke looks like the next spoke, so apparent rotation is only ever known modulo the 45° between them. That is the temporal twin of the 96-pixel ripple above.

Where this breaks

This lesson shows that undersampling substitutes one pattern for another, and gives the arithmetic for where the substitute lands. It does not explain why two samples per cycle is the threshold, or what “average first” is really doing. Both of those are statements about the frequency content of an image, and they belong to the frequency domain unit, which is not written yet. Taking the rule on trust here is deliberate.

Two more limits. The drift table means the “7.385 px” figure is one window’s value, not the table’s; a different crop gives a different period, and any conclusion that depends on the exact number has to name the window. And these code values are not proportional to light (unit 1.2 measured the curve that sits between them), so averaging code values is not averaging light. For a resize nobody notices; for photometry it matters.

Next

Quantization: how many bits a pixel deserves. The samples are in the right places now, so the question becomes how finely each one is allowed to be written down.

Further reading

• Go deeper: Gonzalez & Woods, Digital Image Processing (4th ed.), §2.4 (Image Sampling and Quantization, p. 47) [1]. Spatial and Intensity Resolution (p. 55) and Image Interpolation (p. 61) are the two subsections behind this lesson.
• Related on CondadosAI: what a raw count means · the response curve, which is why averaging code values is not averaging light.

References

[1] Gonzalez, R. C., & Woods, R. E. (2018). Digital Image Processing (4th ed.), §2.4 (Image Sampling and Quantization, p. 47), incl. Spatial and Intensity Resolution (p. 55) and Image Interpolation (p. 61). Pearson. Detailed table of contents.

[2] Nikon. D2X specifications: 23.7 × 15.7 mm CMOS, 12.4 million effective pixels, 4288 × 2848 image area. Product page.

[3] Fairchild, M. D. (2007). The HDR Photographic Survey. Proceedings of the IS&T 15th Color and Imaging Conference, pp. 233–238. doi:10.2352/CIC.2007.15.1.art00044. The scene this unit measures. Used for research and non-commercial publication, as its terms require; images are downloaded, never redistributed.

[4] OpenCV. Geometric image transformations: resize and the interpolation flags, OpenCV 5.0.0 documentation. Docs.