Which residual does the fit minimise?

Eight pixels of a lane dash from a dashcam frame. Least squares with y on x squares vertical gaps, with x on y horizontal ones, and the two disagree by 14° on the same points. Total least squares squares the perpendicular gaps and does not care which axis is which.

Try this

Cycle through the three kinds of residual, then rotate the points and watch the two least-squares lines trade places while the green one follows the dash.

Try it: which residual does the fit minimise?

Where this lab comes from

Fitting a Line to Points: Least Squares vs Total Least Squares

The line you fit to a set of points depends on how you measure the distance to it. Worked by hand on eight pixels of a painted lane dash: least squares y on x gives 18.5°, x on y gives 32.7°, total least squares 20.2°. Over 473 lane markings, the two least-squares lines differ by more than 1° on a third of them.

More in cameras and geometry

Focus, aperture and depth of field — Move the subject and watch the focus plane and the blur follow.
Four ways a fisheye maps angle to radius — Plot r(θ) for the classical designs against a fitted curve.
The scene, and what the camera sees — Orbit a 3-D scene on the left; the pinhole view updates on the right.