Before this unit: Image formation

Homogeneous Coordinates: Why Vision Adds a 1

Lesson 1 of Camera Geometry, and the prerequisite of Image Alignment. Add a third coordinate and a point becomes a ray, a line becomes a 3-vector, and both the line through two points and the point where two lines cross are one cross product. Measured on a pickleball court: a corner 18.5 px outside the frame, found anyway, and two sidelines that are parallel on the ground meeting 2,252 px to the right.

Luis Condados · September 27, 2026

computer-vision geometry homography fundamentals

The court's lines, fitted to paint and extended past the frame: the near corner that falls off the left edge, and the vanishing point far to the right where the two sidelines meet.

Computer vision writes a pixel $(x, y)$ as three numbers, $(x, y, 1)$ . The extra number costs nothing and buys a lot: the line through two points and the point where two lines cross both become one cross product, and lines that are parallel in the world get a place in the picture where they meet.

A pickleball court photographed from behind one corner; the two sidelines and the centre line are extended past the edge of the frame until they meet at one point to the right of the image — Three lines that are parallel on the court, extended in the photograph until they meet, off the frame. Frames from “2026.07.25 WD Open … (Gold Medal match)” by pickleball4you, CC BY 3.0.

You meet this idea well before you meet its name:

Every camera model. The matrix that turns a 3-D point into a pixel only works because points carry that extra coordinate, and the final division by it is the perspective of the photograph [2].
Vanishing points and the horizon. Where parallel lines meet tells you which way the camera points, which is how a photo is levelled or a camera’s tilt is estimated [4].
Finding corners you cannot see. A court, a document or a building corner outside the frame is still the crossing of two lines you can fit, as the worked example below shows.
3-D graphics. Graphics pipelines carry the same extra coordinate for 3-D points and divide by it before drawing.

Write a pixel as three numbers instead of two, $(x, y, 1)$ , and two operations that usually need special cases become one line of code. The line through two points is their cross product. The point where two lines cross is the cross product of the lines. On a pickleball court that finds a corner 18.5 px outside the frame that nobody could click, and puts the point where the two sidelines meet, parallel on the ground, at pixel (2252, 299), off the right edge of a 1920-wide image.

Where we are

Every camera model on this site ends in a division. The pinhole lesson divides by depth, and the camera models post writes the whole camera as one matrix $K[R\,|\,t]$ and then divides by the third component at the end. That third component was never explained. This lesson explains it, on the photograph the image alignment unit is measured on: one frame of a women’s doubles final, taken by a fixed camera behind one corner of the court.

A point is a ray

Take a pixel $(x, y)$ and write it as $(x, y, 1)$ . Now agree that any non-zero multiple names the same pixel: $(2x, 2y, 2)$ and $(x, y, 1)$ are the same point, and to get the pixel back you divide by the last number. The set of all multiples of $(x, y, 1)$ is a line through the origin of 3-D space, so a pixel is a ray, and the image plane is the plane $w = 1$ that every ray crosses once [1].

Almost every ray crosses it. The rays with $w = 0$ run parallel to the plane and never reach it. They are still perfectly good 3-vectors, $(x, y, 0)$ , and they turn out to be the points where parallel lines meet. Hartley and Zisserman call the set of them the line at infinity [1].

Every point on the green-and-amber ray names the same pixel, and dividing by the last coordinate picks the one on the plane. The red ray has

w = 0

and meets the plane nowhere, which is why a vanishing point can be computed but not always drawn.

A line is three numbers, and meet and join are cross products

A line $ax + by + c = 0$ is fixed by $(a, b, c)$ , up to scale, the same way a point is. A point lies on a line when $\mathbf{l}\cdot\mathbf{x} = 0$ . That one equation is symmetric in points and lines, and it gives both constructions for free [1]:

\mathbf{l} = \mathbf{x}_1 \times \mathbf{x}_2 \qquad\qquad \mathbf{x} = \mathbf{l}_1 \times \mathbf{l}_2

The cross product of two vectors is perpendicular to both, so its dot product with each of them is zero. That is the line through both points, or the point on both lines. No slopes, no special case for vertical lines, no special case for parallel ones. Szeliski writes both constructions the same way [2].

Worked example · The near-court corners of frame 45000

Four corners of the near half were found by fitting lines to thousands of paint pixels and intersecting them. Two of them are on the left sideline:

\mathbf{x}_{NBL} = (-18.52,\ 587.39,\ 1) \qquad \mathbf{x}_{NKL} = (669.68,\ 500.09,\ 1)

The first has a negative $x$ : the near-left corner of the court is 18.5 px outside the frame. It was never clicked. It is the cross product of the fitted baseline and the fitted sideline, which do not care where the frame ends.

Join. The left sideline is their cross product:

\mathbf{l}_{left} = \mathbf{x}_{NBL}\times\mathbf{x}_{NKL} = \big(587.39 - 500.09,\ \ 669.68 + 18.52,\ \ (-18.52)(500.09) - (669.68)(587.39)\big) = (87.30,\ 688.20,\ -402{,}625.00)

The right sideline, through $(987.25, 959.91)$ and $(1555.24, 663.22)$ , comes out as $(296.69,\ 567.99,\ -838{,}126.48)$ .

Meet. On the ground the two sidelines are parallel. In the photograph:

\mathbf{x} = \mathbf{l}_{left}\times\mathbf{l}_{right} = (-348{,}111{,}671.0,\ \ -46{,}286{,}369.8,\ \ -154{,}596.5) \;\Rightarrow\; \left(\tfrac{-348{,}111{,}671.0}{-154{,}596.5},\ \tfrac{-46{,}286{,}369.8}{-154{,}596.5}\right) = (2251.8,\ 299.4)

What it means: the sidelines meet 332 px to the right of the frame’s edge and 299 px below its top. That is the vanishing point of the court’s long axis. Do the same on the court itself, in metres, where the sidelines are $y = 0.0254$ and $y = 6.0746$ : the lines are $(0, 1, -0.0254)$ and $(0, 1, -6.0746)$ , their cross product is $(-6.0492,\ 0,\ 0)$ , and $w = 0$ . The same two lines meet at a finite point in the image and at infinity on the ground. The camera moved that point from infinity into the picture.

The whole computation is two cross products and a division:

import numpy as np

nbl, nkl = np.array([-18.52, 587.39, 1.0]), np.array([669.68, 500.09, 1.0])
nbr, nkr = np.array([987.25, 959.91, 1.0]), np.array([1555.24, 663.22, 1.0])

left = np.cross(nbl, nkl)          # the line through two points
right = np.cross(nbr, nkr)
vp = np.cross(left, right)         # the point on two lines
print(left, right, vp[:2] / vp[2]) # ... [2251.8  299.4]

#include <opencv2/core.hpp>
#include <iostream>

int main() {
    cv::Vec3d nbl(-18.52, 587.39, 1), nkl(669.68, 500.09, 1);
    cv::Vec3d nbr(987.25, 959.91, 1), nkr(1555.24, 663.22, 1);
    cv::Vec3d left = nbl.cross(nkl), right = nbr.cross(nkr);
    cv::Vec3d vp = left.cross(right);
    std::cout << vp[0] / vp[2] << ", " << vp[1] / vp[2] << "\n";  // 2251.8, 299.4
}

The horizon is a join of two vanishing points

The baseline and the kitchen line are parallel on the ground too, running across the court. Their image lines meet at $(-1168.4, 161.5)$ , off the left edge this time. Any set of parallel lines on the court meets somewhere on one image line, and joining the two vanishing points gives it: $(0.0403, -0.9992, 208.45)$ once scaled, which crosses the middle column of the frame at row 247. That is the court’s horizon. It runs through the crowd, well above the net, and every direction on the court has its vanishing point somewhere on it [3].

The frame on a wider canvas: the fitted court lines extended past the frame, a red circle on the near-left corner outside the left edge, a white circle on the vanishing point to the right of the frame, and the horizon as a white line through the crowd — The frame (white border) on a canvas wide enough to hold what its lines imply. Green: the sidelines and centre line, meeting at the vanishing point on the right. Amber: the baseline and kitchen line. Red circle: the corner outside the frame. White line: the horizon, the join of the two vanishing points. Frame from “2026.07.25 WD Open … (Gold Medal match)” by pickleball4you, CC BY 3.0.

Now you try

The four white handles are the near-court corners. Each line on screen is the cross product of two of them, and the vanishing point is the cross product of the two sidelines. Drag the far-right corner and watch $w$ . Then press Make sidelines parallel: the image lines become parallel, $w$ goes to zero, and the vanishing point stops being a pixel.

Try it: meet and join on the court

Frame 45000 of "2026.07.25 WD Open … (Gold Medal match)" by pickleball4you, CC BY 3.0.

In the wild

A second court, outdoors, filmed by a different camera for a different tournament [5]. This one is framed tighter, and both near-court corners fall off the picture: the near-left one at $(834.7,\ 1220.2)$ , 140 px below the bottom edge, and the near-right one at $(2039.4,\ 670.6)$ , 119 px past the right edge. Neither could be clicked. Both are cross products of lines fitted to paint that is in the frame, found the same way the worked example found the indoor corner.

The sidelines meet off the left edge this time, at $(-304.6,\ 255.7)$ : the court runs away from this camera towards the left of the frame, where the indoor one ran to the right. Crossing the left sideline with the centre line instead puts the point 54 px away, and seen from the kitchen corner the two directions differ by 1.05°, three times the indoor 0.36°. The outdoor lines are shorter in the frame and the lens is wider, and both make the angle harder to pin down.

Where this breaks

A vanishing point magnifies small angle errors. The centre line is parallel to both sidelines on the ground, so all three should meet at one image point. Its own vanishing point, crossed with the left sideline, lands at $(2344.3, 287.7)$ , 93 px from the one above. Seen from the kitchen corner of the centre line, the two directions differ by 0.36°. The lines agree well, and the point where they meet is far away, and far away is where a small angle becomes a long distance. Estimating vanishing points robustly, from many lines at once, is its own problem [4]. The lens contributes: the near baseline bows by 7.7 px across the frame, and a line fitted to a bowed stripe is slightly rotated. The unit hub measures that bend.

A representation is not a map. Everything above found points and lines inside one image. Nothing yet says which pixel is which point on the court, in metres. That needs a transformation from one plane to the other, and a 3×3 matrix acting on these same 3-vectors is the right size for it.

The image alignment unit builds that matrix, starting with what a 2×2 can and cannot do, and uses this lesson’s cross products in every step. The rest of unit 4.1 is in camera models, calibration and PnP, where the camera’s own 3×4 matrix acts on the 4-vector version of the same idea.

Run it: every code block on this page has a cell in the unit’s notebook, open it in Colab.

References

[1] Hartley, R., & Zisserman, A. (2004). Multiple View Geometry in Computer Vision (2nd ed.), §2.2 “The 2D projective plane”, p. 26. Cambridge University Press.

[2] Szeliski, R. (2022). Computer Vision: Algorithms and Applications (2nd ed.), §2.1 “Geometric primitives and transformations”. Springer. Free PDF

[3] Hartley, R., & Zisserman, A. (2004). Multiple View Geometry in Computer Vision (2nd ed.), §8.6 “Vanishing points and vanishing lines”, p. 213. Cambridge University Press.

[4] Szeliski, R. (2022). Computer Vision: Algorithms and Applications (2nd ed.), §7.4 “Lines and vanishing points”. Springer. Free PDF

[5] pickleball4you (2024). 2024.08.30 MS4.0 Philip Wong vs Tristan Clark (Round Robin, match 6). YouTube, CC BY 3.0. youtube.com/watch?v=K0qrASvix3Y

Tagged