Before this unit: Homogeneous coordinates
Rotate, Scale, Shear: Images as 2×2 Matrices
Lesson 1 of Image Alignment. A 2×2 matrix is where it sends the two axes; its determinant is how much it scales area and its SVD is rotate, stretch, rotate. The one thing it cannot do is make parallel lines meet. On a pickleball court, the sidelines are 20.35° apart in the photograph, so the best 2×2 misses every corner by 60 px.
Rotating, zooming or slanting a picture moves every pixel by the same small recipe: two numbers say where one step to the right ends up, two say where one step up ends up. Those four numbers are a 2×2 matrix, and they are all a rotation, a zoom or a slant ever is.

Where 2×2 transforms do the work:
- Photo editors. Rotate, resize and skew are each one 2×2 matrix applied to every pixel [1].
- Training data. The random rotations, zooms and shears used to augment images for a neural network are 2×2 matrices chosen at random.
- Slanted text. A synthetic italic is a shear: every row moves sideways in proportion to its height.
- Scanned pages. A page that went into the scanner slightly rotated is straightened by the inverse rotation.
What a 2×2 cannot do is the reason the rest of this unit exists, and a photograph of a court shows it.
A 2×2 matrix can rotate, stretch and slant a picture, but it can never make two parallel lines meet, so it cannot map a court seen from a corner. In the photograph the court’s sidelines are 20.35° apart, and the best 2×2 anyone can fit misses every corner by 60.4 px.
Where we are
Lesson 1 of image alignment, and the first half of step 5 of its pipeline: choosing the map. The unit measures everything on one frame of a pickleball final, and the question is how to map the court, in metres, onto the photograph, in pixels. The homogeneous coordinates lesson found the court’s lines and corners in the image (steps 2 and 3). This lesson tries the simplest possible map between the two and finds out why it is not enough.
Two arrows decide everything
The simplest rule that could send court metres to pixels mixes the two coordinates with four fixed numbers, one for each pair of input and output axis. Multiply a point by a matrix and you get a new point:
The right-hand side is the whole idea. The first column is where the point lands and the second is where lands; every other point is a mix of the two. Rotation, uniform scale, stretching one axis and shearing are all special choices of those two columns [1], and Gonzalez and Woods list the same family as the basic geometric transformations of an image [2].
Two numbers read off any 2×2 tell you what it does.
The determinant is the area of the parallelogram the two columns span. A unit square goes in; a parallelogram of area comes out. Negative means the map mirrors, zero means it squashes the plane onto a line.
The singular value decomposition writes any 2×2 as : rotate, stretch by along one axis and along the other, rotate again. No 2×2 does anything that is not those three steps.
The one thing a 2×2 cannot do
Take a line through with direction : the points . Its image is , a line with direction . A second line with the same direction maps to a line with the same direction . Parallel lines stay parallel under any 2×2, whatever its four numbers are.
The two sidelines of a court are parallel. In frame 45000 they are 20.35° apart, and the previous lesson found where they meet. So no 2×2 can be the map from the court to this photograph. The only question left is how wrong the best one is.
import numpy as np
court = np.array([[0.0254, 0.0254], [0.0254, 6.0746], [4.6004, 0.0254], [4.6004, 6.0746]]) # m
image = np.array([[-18.52, 587.39], [987.25, 959.91], [669.68, 500.09], [1555.24, 663.22]]) # px
c0, i0 = court - court.mean(0), image - image.mean(0)
M = np.linalg.lstsq(c0, i0, rcond=None)[0].T # image ≈ M @ court, both centred
U, s, Vt = np.linalg.svd(M)
print(M.round(2), np.linalg.det(M).round(0), s.round(1))
print(np.linalg.norm(c0 @ M.T - i0, axis=1).round(1)) # [60.4 60.4 60.4 60.4]#include <opencv2/core.hpp>
#include <iostream>
int main() {
cv::Mat court = (cv::Mat_<double>(4, 2) << 0.0254, 0.0254, 0.0254, 6.0746, 4.6004, 0.0254, 4.6004, 6.0746);
cv::Mat image = (cv::Mat_<double>(4, 2) << -18.52, 587.39, 987.25, 959.91, 669.68, 500.09, 1555.24, 663.22);
cv::Mat c0 = court.clone(), i0 = image.clone();
for (int j = 0; j < 2; ++j) { c0.col(j) -= cv::mean(court.col(j))[0]; i0.col(j) -= cv::mean(image.col(j))[0]; }
cv::Mat Mt;
cv::solve(c0, i0, Mt, cv::DECOMP_SVD); // least squares: c0 * Mt = i0
cv::Mat M = Mt.t(), w, u, vt;
cv::SVD::compute(M, w, u, vt);
std::cout << M << "\ndet " << cv::determinant(M) << "\nsigma " << w.t() << "\n";
}
Now you try
The four sliders are , , and . The red and blue arrows are the two columns, the grey court is the input and the coloured one is the output. Try the presets, then chase the one thing the readout says never changes: the angle between the two green sidelines. Best fit to the photo loads the matrix from the worked example, divided by 100 so it fits on screen.
How close is this to OpenCV?
The worked example is three lines of NumPy, and OpenCV has a call for each thing it
claims: cv2.transform moves points through a matrix, cv2.determinant and
cv2.SVDecomp take it apart. The cells use the four corners as printed above.
import numpy as np
import cv2
# the near-half corners: metres and pixels
court = np.array([[0.0254, 0.0254], [0.0254, 6.0746],
[4.6004, 0.0254], [4.6004, 6.0746]])
image = np.array([[-18.52, 587.39], [987.25, 959.91],
[669.68, 500.09], [1555.24, 663.22]])
# the best 2x2, with both sets centred
c0 = court - court.mean(0)
i0 = image - image.mean(0)
M = np.linalg.lstsq(c0, i0, rcond=None)[0].T
det = M[0, 0] * M[1, 1] - M[0, 1] * M[1, 0]
sigma = np.linalg.svd(M, compute_uv=False)
print(M.round(2))
print("det:", round(det), " sigma:", sigma.round(1))[[137.29 156.33] [-41.97 44.27]] det: 12639 sigma: [208.1 60.7]
# OpenCV moves the points and takes the matrix apart
pts = c0[None].astype(np.float32)
moved = cv2.transform(pts, M.astype(np.float32))[0]
print("max diff, cv2.transform:",
round(float(np.abs(moved - c0 @ M.T).max()), 4))
print("cv2.determinant:", round(cv2.determinant(M)))
w, u, vt = cv2.SVDecomp(M)
print("cv2.SVDecomp:", w.ravel().round(1))
miss = np.linalg.norm(c0 @ M.T - i0, axis=1)
print("miss at each corner (px):", miss.round(1))max diff, cv2.transform: 0.0 cv2.determinant: 12639 cv2.SVDecomp: [208.1 60.7] miss at each corner (px): [60.4 60.4 60.4 60.4]
The two agree to the last printed digit, and the miss is the worked example’s: 60.4 px at every corner, whatever library computes it.
In the wild
The same test on the outdoor court in the homogeneous coordinates lesson, filmed by a different camera [3]. Its sidelines are 30.21° apart in the image, half as much again as the indoor court’s 20.35°. The best 2×2 does correspondingly worse: 101.1 px of error at each corner against 60.4 indoors. Two courts are not a trend, but they point the same way: the further apart the sidelines are in the image, the further any linear map falls short.
Where this breaks
Two limits, and both are visible above.
It cannot move the origin. for every matrix. The worked example only worked because both point sets were centred first, which is a translation done by hand, outside the matrix. Any real alignment needs the shift inside the model.
It cannot make parallel lines meet. No choice of four numbers turns the court’s parallel sidelines into the converging ones in the photograph, and adding a translation will not change that either: a shift moves both lines by the same amount and leaves them parallel. The best linear map is off by 60.4 px everywhere, and the remaining error is not noise to be averaged away. It is the wrong shape of model.
Exercises
- Area on the court, area in the photo. Write
area_scale(M), the number of pixels one square metre of court covers under the 2×2Mfrom the cells above.
def area_scale(M):
return 0.0 # pixels per square metre
print(area_scale(M))- Predict, then run. Push the direction of a sideline, in court metres, through
M, and measure the two sidelines in the photo from the corners. Before running, say how many different directionsMgives the two sidelines. - Two quick determinants. What is the determinant of a 30° rotation, and of
2 * M?
What you should see
These come from the “Lesson 1” cells of the unit notebook, image_alignment.ipynb, and are
the same on OpenCV 4.11 (this page) and 5.0.
- About 12,639 px² per m², the worked example’s determinant.
- One:
Msends both sidelines to −17.0°. In the photo they run at −7.23° and −27.58°, the 20.35° apart this lesson starts from. - 1, because a rotation keeps area; and 50,556, four times
M’s, because doubling both columns doubles both sides of the parallelogram.
Check yourself
Four questions on this lesson. Pick an answer to see why it is right or wrong.
What do the two columns of a 2×2 matrix tell you?
The first column is where (1, 0) goes and the second where (0, 1) goes. A 2×2 cannot move the picture at all: it always sends (0, 0) to (0, 0).
The best 2×2 for the near half of the court has determinant 12,639. What does that number say?
The determinant is the area scale. A 2×2 has one for the whole plane, while the true scale is 62,697 px² per m² at the near-right corner and 10,479 at the kitchen corner, so it is wrong at both ends.
Why can no 2×2 map the court onto this photograph?
A line with direction d goes to a line with direction M d, so two parallel lines stay parallel. The best 2×2 misses every corner by 60.4 px.
What is the determinant of a pure 30° rotation?
A rotation keeps area, so its determinant is 1. Doubling a matrix multiplies its determinant by four.
Next
Both limits are fixed by the same trick from the homogeneous coordinates lesson: write points as and use a 3×3. Its last column carries the translation, and letting its last row vary lets parallel lines meet. Lesson 2 adds one at a time and measures what each buys on this court.
Run it: every code block on this page has a cell in the unit’s notebook, open it in Colab.
References
[1] Szeliski, R. (2022). Computer Vision: Algorithms and Applications (2nd ed.), §2.1.1 “2D transformations”. Springer. Free PDF
[2] Gonzalez, R. C., & Woods, R. E. (2018). Digital Image Processing (4th ed.), §2.6 “Introduction to the Basic Mathematical Tools Used in Digital Image Processing”, Geometric Transformations, p. 84. Pearson.
[3] pickleball4you (2024). 2024.08.30 MS4.0 Philip Wong vs Tristan Clark (Round Robin, match 6). YouTube, CC BY 3.0. youtube.com/watch?v=K0qrASvix3Y