Edge Detection: Finding Where One Thing Stops and Another Begins

Finding where one thing stops and another begins, and turning those places into something a computer can point at.

That is the whole unit. Five lessons: what counts as an edge, the two derivatives that can find one, why the obvious way of using them does not work, the repair that made Canny’s method the default for forty years, and the step that turns a curve into a point you can match against another photo.

Before this

You need 2D convolution. Every operator here is a small kernel slid over the image, and the separability trick from that post is what makes the 3×3 gradient operators cheap. If the image is still a black box, start at how a camera forms an image and the image as data.

The photograph everything is measured on

Every lesson works on one image: view 0 of the Middlebury templeRing set [6], a plaster temple against a black backdrop, 640×480. It is the same photograph the SIFT post measures on, rotated upright, contrast-lifted and resized to 360×480 once. Using one image the whole way through means a number from lesson 2 can be compared against a number from lesson 5 without re-anchoring.

Three places in that frame carry the unit:

• the black background, where nothing changes in any direction;
• a column’s silhouette against it, where brightness changes across the edge and not along it;
• the corner where the base meets a column, where it changes in both directions at once.

Tracing the silhouette with a pencil is edge detection. The background has nothing to trace. The corner is where two edges meet, which turns out to be a different problem with a different answer, and that is the last lesson.

Topics

1. What is an edge?. If you had to define one with arithmetic instead of a pencil, what would you write? Step, ramp and roof profiles, and the fact that an edge is a peak in the first derivative and a zero-crossing in the second [1].
2. Gradients: central differences, Prewitt, Sobel. How do you measure that peak, and why do three operators that answer the same question behave so differently on a noisy photo?
3. The Laplacian, LoG and zero-crossings. The second derivative gives thin contours that close into loops, which sounds better. What does Marr and Hildreth’s method cost you [3], and why does it need smoothing before it is usable at all?
4. Canny. Thresholding a gradient gives edges that are too thick, then too broken, and never both right. Canny’s answer was two separate ideas: thin the ridge, then decide membership by connection [2].
5. Corners: the structure tensor. A contour tells you where a boundary runs but not where you are along it. Two eigenvalues fix that [4], and they are where the SIFT unit begins.

The order is a ramp, not a list. Each lesson ends at the thing the next one repairs, so lesson 3 is where the naive use of a gradient falls over and lesson 4 is the repair.

How they fit together

What the unit measures

Every lesson carries its own numbers. Three results shape the order the lessons are in, all computed on the canonical frame and saved in output/edge_numbers.json:

Result	Measurement
Averaging across rows is what buys noise robustness	At noise $\sigma = 20$, a plain central difference recovers 0.6262 of its clean edge set; Prewitt recovers 0.7968 and Sobel 0.7814
Sobel is not the better operator for noise	Prewitt leads Sobel at every noise level tested, by 3–5× the spread across five runs
Linking beats thresholding at equal cost	At 3,810 edge pixels, hysteresis returns 204 contours averaging 18.68 px; a single threshold keeping the same pixels returns 393 averaging 9.69 px
Smoothing is what makes zero-crossings usable	Noise multiplies the crossing pixels by 2.76× at $\sigma = 1.0$ and by 1.02× at $\sigma = 3.0$

The Prewitt result has a mechanism rather than a coincidence behind it, and lesson 2 works it out: for the same three-row support, Prewitt’s normalised weights have $\sum w^2 = 1/3$ against Sobel’s $3/8$, so Prewitt passes about 11% less noise power. What Sobel’s $[1\,2\,1]$ buys is rotational symmetry.

Reproducibility

Parameter	Value
CPU	12th Gen Intel Core i7-12700H, 20 threads
GPU	none used; every operator here runs on the CPU
RAM / OS	31 GB · Ubuntu 22.04.5 LTS, kernel 6.8.0-136
Key versions	Python 3.12.9, OpenCV 5.0.0 [7], NumPy 2.5.2
Data	Middlebury templeRing view 0 [6], 640×480, preprocessed to 360×480
Commands	uv run edge-download then uv run edge-experiments
Runs	noise results are the mean of 5 realizations from seed 20260818; the standard deviation of each is in the JSON
Excluded	no timing is reported in this unit; every number is an accuracy or a count

Every number in the table above is a key in output/edge_numbers.json.

The lessons carry no reproducibility table of their own, and now each of them also repeats its measurement on a photograph rather than only on the unit’s frame. A temple cloister gives edges a median width of 2.30 px — not one of the 400 measured is a step — and puts 65.5% of its strong gradients into two orientation bands. An 1814 newspaper shows 96.3% of the Laplacian’s zero-crossings sitting where nothing is changing. Hysteresis on the cloister returns 654 long contours out of 4,340, against 73,025 fragments from the low threshold alone. And Harris corners on a market stall come back 85–88% of the time under rotation. The photographs are Wikimedia Commons files under CC0, registered in CondadosAI/cv-assets with verified licences and a sha256 each, downloaded and never redistributed.

Limitations & caveats

• One image. Everything above is measured on a single 360×480 photograph of a plaster temple. The Prewitt-over-Sobel ordering held at every noise level tested on that image, which is enough to rule out variance and not enough to make it a claim about photographs in general.
• There is no ground truth here. “Agreement” means agreement with the same operator’s own result on the clean image. It measures stability under noise, not correctness. No human labelled these edges, and an operator that is confidently wrong in the same place twice scores perfectly.
• The noise is synthetic. Gaussian noise added in software is not sensor noise. Real read noise is signal-dependent, and demosaicing correlates it between neighbouring pixels, which is the structure these operators average over.
• The 5% edge budget is a choice. It makes operators with different response scales comparable, and it also fixes how much of the image is called an edge. A different budget moves the absolute numbers.
• Nothing is timed. All four operators cost a 3×3 convolution and the differences would be noise on any machine that matters.

Where this lands

Lesson 5 hands its two eigenvalues straight to SIFT, which asks what happens to them when the camera moves closer. The contours from lesson 4 are the input the next unit, boundary detection, fits lines and curves to. Gonzalez and Woods put the Hough transform in the same section as Canny for that reason [1].

Further reading

• Go deeper: Gonzalez & Woods, Digital Image Processing (4th ed.), §10.2 “Point, Line, and Edge Detection” [1]. It covers this unit end to end, including the Hough transform we defer to the next one.
• The wider map: Szeliski, ch. 7 “Feature Detection and Matching” [5], for where edges sit relative to everything built on them.
• Related on CondadosAI: 2D convolution is the prerequisite · SIFT is the sequel, and it is where these corners stop being points and start being descriptors.

References

[1] Gonzalez, R. C., & Woods, R. E. (2018). Digital Image Processing (4th ed.), ch. 10 “Image Segmentation I: Edge Detection, Thresholding, and Region Detection”, §10.2 “Point, Line, and Edge Detection”, pp. 763–804. Pearson.

[2] Canny, J. F. (1986). A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6), 679–698. doi:10.1109/TPAMI.1986.4767851

[3] Marr, D., & Hildreth, E. (1980). Theory of edge detection. Proceedings of the Royal Society of London B, 207(1167), 187–217. doi:10.1098/rspb.1980.0020

[4] Harris, C. G., & Stephens, M. (1988). A Combined Corner and Edge Detector. Proceedings of the Alvey Vision Conference 1988, pp. 1–6. doi:10.5244/C.2.23

[5] Szeliski, R. (2022). Computer Vision: Algorithms and Applications (2nd ed.), ch. 7 “Feature Detection and Matching”. Springer. Free PDF

[6] Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. CVPR 2006, pp. 519–528. doi:10.1109/CVPR.2006.19 — the Middlebury multi-view datasets: vision.middlebury.edu/mview

[7] OpenCV Documentation (5.0). Image Processing — Canny Edge Detector. docs.opencv.org/5.0