How 2D Convolution Works on Images: Kernels, Filters, and the Math, Interactively

Almost everything you do to an image — blurring it, sharpening it, finding its edges — is the same operation with a different 3×3 grid of numbers. And the convolutional layers that power modern vision models are that exact operation, with the grids learned instead of hand-picked. Convolution is worth understanding in your bones. So instead of just writing the formula, let’s slide a kernel across a grid and watch the math happen, then run real kernels on a real image.

This is the natural next step after how a camera turns light into an array of numbers: now that the image is a grid of integers, convolution is how we transform it.

1. The whole operation, one window at a time

Convolution slides a small grid of weights — the kernel — over the image. At each position it multiplies the nine overlapping numbers by the nine kernel weights, adds them up, and writes that single number to the output. Multiply, accumulate, move one pixel, repeat.

That’s it. Press Play and watch the kernel sweep across the input; the readout underneath shows the exact multiply-accumulate for the current window. Try swapping the kernel (Sobel X lights up the vertical edge; Box blur averages; Identity copies) and the input pattern.

A few things to notice while it runs:

• The output is smaller than the input. An 8×8 image with a 3×3 kernel gives a 6×6 output, because the window can’t hang off the edge. We fix that with padding below.
• The kernel decides everything. Same sliding mechanic, completely different result — a blur, an edge map, a sharpen — purely from the nine weights.
• Edges produce big responses. Where the input jumps from dark to bright, an edge kernel’s positive and negative weights stop cancelling and the output spikes.

2. The math (and the one detail everyone gets wrong)

Write the image as $I$ and a kernel $K$ of size $(2a{+}1)\times(2b{+}1)$. What the demo above actually computes is cross-correlation:

True convolution flips the kernel in both directions first:

The flip is the only difference. For symmetric kernels (blur, Gaussian, Laplacian) the two are identical, which is why the distinction is easy to ignore. It matters for asymmetric kernels like Sobel or emboss, where convolution gives the mirror of correlation. The practical catch worth memorizing:

OpenCV’s filter2D and the “convolution” in every deep-learning framework actually compute cross-correlation — no flip. Since a CNN learns its weights, the flip would just learn the mirrored kernel and the network can’t tell the difference. Only reach for a real flip when you’re matching a textbook convolution exactly.

3. Padding, stride, and output size

Two knobs control the output’s shape.

Padding ($p$) adds a border of pixels around the input so the kernel can be centered on edge pixels. With $p = 1$ for a 3×3 kernel, the output is the same size as the input (“same” padding); with $p = 0$ it shrinks (“valid” padding — what the demo shows).

Stride ($s$) is how far the window jumps each step. Stride 1 visits every pixel; stride 2 skips every other one, halving the output’s width and height (this is how convolutional layers downsample).

For an input of size $H$, kernel size $k$, padding $p$, and stride $s$, the output side length is:

Quick check: $H=8$, $k=3$, $p=0$, $s=1$ gives $\lfloor 5/1 \rfloor + 1 = 6$ — exactly the 6×6 output you saw. Want the output the same size as the input? Pick $p = (k-1)/2$.

4. Kernels are filters — feel what each one does

The grid of weights is the filter. Below, the same convolution runs on a real (if synthetic) image. Click a preset, or edit the nine numbers and the divisor by hand and watch the result update live. Toggle grayscale to see edge kernels more clearly, or upload your own image.

What the classic 3×3 kernels do, and why:

• Box blur $\frac{1}{9}\begin{bmatrix}1&1&1\\1&1&1\\1&1&1\end{bmatrix}$ — replaces each pixel with the average of its neighborhood. The divisor keeps brightness constant. Smooths noise, but produces hard, boxy artifacts.
• Gaussian blur $\frac{1}{16}\begin{bmatrix}1&2&1\\2&4&2\\1&2&1\end{bmatrix}$ — a weighted average that falls off with distance. Smoother and more natural than a box; the standard pre-blur before edge detection.
• Sharpen $\begin{bmatrix}0&-1&0\\-1&5&-1\\0&-1&0\end{bmatrix}$ — boosts the center and subtracts its neighbors, amplifying local contrast. It’s the identity plus a Laplacian.
• Sobel X / Y — estimate the image gradient in one direction. They respond to intensity changes, i.e. edges, and are the front end of classic edge detectors.
• Laplacian $\begin{bmatrix}0&1&0\\1&-4&1\\0&1&0\end{bmatrix}$ — a second derivative that fires on edges in every direction at once.

Notice the weights of every blur kernel sum to 1 (after the divisor), so they preserve overall brightness; every edge kernel sums to 0, so flat regions go to zero and only changes survive. That sum is a quick tell for what a kernel does.

5. Separable kernels: the same result, far cheaper

A 2D kernel is separable if it’s the outer product of two 1D kernels, $K = \mathbf{u}\,\mathbf{v}^{\top}$. When it is, you can convolve with $\mathbf{u}$ along the rows and $\mathbf{v}$ down the columns and get the identical answer — for a $k \times k$ kernel that’s $2k$ multiplies per pixel instead of $k^2$.

The Sobel X kernel is exactly this — a smoothing column times a differencing row:

The Gaussian is separable too. For a 5×5 kernel that’s 10 multiplies per pixel instead of 25 — a 2.5× saving that grows with kernel size, which is why every production blur is implemented this way.

6. What happens at the border

The window falls off the edge of the image, so something has to fill the missing pixels. The common choices:

• Zero padding — treat outside pixels as 0. Simple, but darkens and can create a faint frame.
• Replicate — extend the edge pixel outward. A safe default (it’s what the demo above uses).
• Reflect — mirror the image across the border. Usually the most artifact-free for blurs.

It only affects a one-pixel-wide rim for a 3×3 kernel, but for big blurs the choice is visible — pick reflect when you care.

7. Doing it for real with OpenCV

For an arbitrary kernel, use filter2D (remember: it’s cross-correlation). For the common filters, reach for the purpose-built, separable, and SIMD-optimized functions instead of rolling your own.

import cv2
import numpy as np

img = cv2.imread("input.jpg")

# --- arbitrary kernel -------------------------------------------------
sharpen = np.array([[ 0, -1,  0],
                    [-1,  5, -1],
                    [ 0, -1,  0]], dtype=np.float32)
sharpened = cv2.filter2D(img, ddepth=-1, kernel=sharpen)   # cross-correlation

# For a TRUE (flipped) convolution, flip the kernel first:
true_conv = cv2.filter2D(img, -1, cv2.flip(sharpen, -1))

# --- purpose-built, separable, optimized ------------------------------
blurred = cv2.GaussianBlur(img, (3, 3), sigmaX=0)            # separable internally
gx = cv2.Sobel(img, cv2.CV_32F, 1, 0, ksize=3)              # vertical edges
gy = cv2.Sobel(img, cv2.CV_32F, 0, 1, ksize=3)              # horizontal edges
edges = cv2.Laplacian(img, cv2.CV_32F, ksize=3)

# Control the border explicitly:
soft = cv2.GaussianBlur(img, (9, 9), 0, borderType=cv2.BORDER_REFLECT)

#include <opencv2/opencv.hpp>

cv::Mat img = cv::imread("input.jpg");

// --- arbitrary kernel -------------------------------------------------
cv::Mat sharpen = (cv::Mat_<float>(3, 3) <<
     0, -1,  0,
    -1,  5, -1,
     0, -1,  0);
cv::Mat sharpened;
cv::filter2D(img, sharpened, -1, sharpen);          // cross-correlation

// For a TRUE (flipped) convolution, flip the kernel first:
cv::Mat flipped, trueConv;
cv::flip(sharpen, flipped, -1);
cv::filter2D(img, trueConv, -1, flipped);

// --- purpose-built, separable, optimized ------------------------------
cv::Mat blurred, gx, gy, edges, soft;
cv::GaussianBlur(img, blurred, cv::Size(3, 3), 0);  // separable internally
cv::Sobel(img, gx, CV_32F, 1, 0, 3);                // vertical edges
cv::Sobel(img, gy, CV_32F, 0, 1, 3);                // horizontal edges
cv::Laplacian(img, edges, CV_32F, 3);

// Control the border explicitly:
cv::GaussianBlur(img, soft, cv::Size(9, 9), 0, 0, cv::BORDER_REFLECT);

// No OpenCV — the sliding multiply-accumulate from §1, by hand.
// img: flat H*W*3 BGR bytes; W, H known. Needs <vector> and <algorithm>.
const float K[3][3] = {{ 0, -1,  0},
                       {-1,  5, -1},
                       { 0, -1,  0}};            // sharpen

auto at = [&](int y, int x, int c) -> unsigned char {
    x = std::clamp(x, 0, W - 1);                 // replicate border
    y = std::clamp(y, 0, H - 1);
    return img[(y * W + x) * 3 + c];
};

std::vector<unsigned char> out(H * W * 3);
for (int y = 0; y < H; ++y)
    for (int x = 0; x < W; ++x)
        for (int c = 0; c < 3; ++c) {
            float acc = 0.f;
            for (int i = -1; i <= 1; ++i)        // cross-correlation: no flip
                for (int j = -1; j <= 1; ++j)
                    acc += K[i + 1][j + 1] * at(y + i, x + j, c);
            // edge kernels go negative — clamp back into [0,255] for display.
            out[(y * W + x) * 3 + c] =
                static_cast<unsigned char>(std::clamp(acc + 0.5f, 0.f, 255.f));
        }

// True convolution? Flip the kernel (i -> -i, j -> -j) before summing.
// Box blur, Gaussian, Sobel, and Laplacian are this SAME loop, other weights.

A practical note on depth: edge kernels produce negative numbers. If you write them into an 8-bit image they get clamped to 0 and you lose half the gradient — so compute into a signed/float type (CV_32F), then take the absolute value or rescale before display. That’s the same clamping you see when the demo’s Sobel output hits its floor.

8. From hand-picked kernels to learned ones

Here’s the payoff. For decades, computer vision meant designing these kernels by hand — Sobel for edges, Gaussian for smoothing, Gabor for texture. A convolutional neural network [3] keeps the exact operation you’ve been watching and throws away the hand-design: each conv layer is a stack of kernels whose weights are learned by gradient descent, and stacking them lets early layers find edges while deeper ones compose those into textures, parts, and objects.

So when you read that a model has a “3×3 conv, stride 2, padding 1,” you now know exactly what that means down to the arithmetic — it’s the sliding multiply-add from §1, with the output size from §3, and weights that were learned instead of typed. That’s the same machinery inside the detectors we benchmark on an Intel iGPU in RF-DETR vs YOLO-NAS at the edge.

Limitations & caveats

• “Convolution” here is cross-correlation. OpenCV’s filter2D and every deep-learning framework skip the kernel flip — fine for symmetric kernels and for CNNs (the flip is just absorbed into learned weights), but it gives the mirror of a textbook convolution for asymmetric kernels like Sobel [1].
• Border handling changes the output. Zero/replicate/reflect padding each produce different rim pixels; for large blurs the choice is visible. Pick it explicitly rather than trusting the default.
• Edge kernels go negative. Writing them into uint8 clamps half the gradient to zero — compute in a signed/float type, then take the absolute value or rescale.
• Separability is only an optimization for separable kernels. Box and Gaussian factor into 1-D passes ($k^2 \to 2k$); a general learned or arbitrary kernel does not, so the speed-up isn’t free everywhere [2].

Takeaways

• Convolution is one tiny loop: slide a kernel, multiply the overlap, sum, write one pixel. Everything else is bookkeeping.
• The kernel is the filter. Blur, sharpen, and edge detection differ only in nine numbers; blur kernels sum to 1, edge kernels sum to 0.
• “Convolution” in OpenCV and deep learning is really cross-correlation — no kernel flip. It only matters for asymmetric kernels.
• Output size is $\lfloor (H + 2p - k)/s \rfloor + 1$. Use $p=(k-1)/2$ to keep the size; use stride to downsample.
• Separable kernels turn $k^2$ multiplies into $2k$ — always prefer OpenCV’s built-in GaussianBlur / Sobel over a manual filter2D.
• A CNN is this same operation with learned weights — understand convolution by hand and conv layers stop being a black box.

Further reading

• Go deeper on CNNs: Goodfellow et al., Deep Learning ch. 9 — from this exact operation up to convolutional architectures.
• Convolution arithmetic, animated: Dumoulin & Visin’s visual guide — padding/stride/dilation as GIFs.
• Related on CondadosAI: the array convolution operates on; learned kernels in production in the edge detector benchmark.

References

[1] Gonzalez, R. C., & Woods, R. E. (2018). Digital Image Processing (4th ed.), ch. 3 (spatial filtering & correlation vs convolution). Pearson.

[2] Szeliski, R. (2022). Computer Vision: Algorithms and Applications (2nd ed.), §3.2 (linear filtering & separable filters). Springer. Free PDF.

[3] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning, ch. 9 (convolutional networks). MIT Press. Free online.

[4] Dumoulin, V., & Visin, F. (2016). A Guide to Convolution Arithmetic for Deep Learning. arXiv:1603.07285. (Padding/stride/output-size formulas.)

[5] OpenCV. Image Filtering — filter2D, GaussianBlur, Sobel, Laplacian. Docs.