Learn & Understand

The Sliding Window: How Convolution Lets Machines See

The convolution output size calculator computes how the dimensions of an image change as it passes through a convolutional layer. Convolution is the operation at the heart of how machines process images, and it works by sliding a small window across the image to detect local features. Understanding how this sliding window operates, and why it revolutionized machine vision, reveals the elegant idea that lets neural networks recognize the visual world, and why the output dimensions matter for designing such networks.

Looking at Small Patches

Rather than examining an entire image at once, a convolutional layer looks at small patches of it, one region at a time. A little window, called a filter or kernel, is slid across the image, and at each position it examines the small patch beneath it, computing a value that indicates whether a particular local feature, an edge, a texture, a spot of colour, is present there. By sliding across the whole image, the filter builds a map of where that feature appears. This is convolution: a small detector sweeping across the image.

The Same Detector Everywhere

A powerful aspect of convolution is that the same filter is applied at every position, so a feature is detected the same way wherever it appears in the image. An edge detector finds edges in the corner as readily as in the center, because the identical window sweeps over the whole image. This sharing means the network does not have to learn to recognize a feature separately for every location; it learns one detector and applies it everywhere. This efficiency was a key insight that made image processing by neural networks practical.

Why the Output Shrinks

As a filter sweeps across an image, the resulting map of feature locations is typically a bit smaller than the original, because the window cannot extend past the edges without running off. How much it shrinks depends on the filter's size, how far it moves between positions, and whether the image is padded with a border to preserve its dimensions. These choices determine the output size, and getting them wrong can shrink a feature map away to nothing across many layers, a real design pitfall the calculator helps avoid.

Building Vision Layer by Layer

Convolutional layers are stacked, each detecting features in the output of the last, so that simple features like edges combine into more complex ones like shapes and eventually whole objects. Tracking how the dimensions change through this stack is essential to designing a working network, which is exactly what the calculator computes from the layer's settings. Convolution, the humble sliding window, is what gave machines the ability to see, and the calculator's output-size arithmetic is a practical necessity for anyone assembling the layered vision systems built upon it.

See how much of the image influences a deep neuron with the CNN Receptive Field Calculator, or count the compute with the Neural Network FLOPs Calculator.