18.2 Convolutional Neural Networks (CNNs) II: Pooling and Architectures

After applying a convolution and an activation function to an image, a CNN typically uses a pooling layer to downsample the feature map. This process is crucial for making the network more manageable and robust.

The Pooling Layer

A pooling layer works much like a convolutional layer, sliding a window over its input. However, instead of performing a weighted sum, it performs a fixed statistical operation. The two most common types of pooling are:

Max Pooling: For each window, the maximum value is taken. This is the most common type of pooling. It effectively asks, "Was this feature detected in this neighborhood?" and is good at preserving the most prominent features.
Average Pooling: For each window, the average of all values is taken. This provides a more smoothed-out, summary representation of the features in a neighborhood.

The visualization below shows a 2x2 max pooling operation with a stride of 2 being applied to a 4x4 feature map.

Max Pooling Visualization

Input Feature Map

→

Pooled Feature Map

Why Use Pooling?

Dimensionality Reduction: Pooling reduces the spatial dimensions (width and height) of the feature maps, which decreases the number of parameters and computational cost in subsequent layers.
Translation Invariance: By summarizing a neighborhood, pooling makes the representation slightly more robust to small shifts and distortions in the input image. The exact location of a feature becomes less important than its rough location relative to other features.

Classic CNN Architectures

A complete CNN architecture is formed by stacking convolutional layers, activation functions (like ReLU), and pooling layers, followed by one or more fully connected layers for final classification.

LeNet-5

One of the earliest successful CNNs, proposed by Yann LeCun in 1998 for handwritten digit recognition. Its architecture is simple but established the foundational pattern for modern CNNs.

Structure: [INPUT] → CONV → POOL → CONV → POOL → FC → FC → [OUTPUT]

LeNet-5 Diagram:

Input (32x32) → C1: 6 filters (5x5) → S2: Avg Pool (2x2) → C3: 16 filters (5x5) → S4: Avg Pool (2x2) → C5: FC (120 units) → F6: FC (84 units) → Output

AlexNet

The model that kickstarted the deep learning revolution by winning the 2012 ImageNet competition with a dramatically lower error rate than previous methods. It was much deeper and larger than LeNet-5 and was the first to use ReLU activations and dropout for regularization.

Structure: A much deeper stack of CONV, POOL, and FC layers, designed to run on two GPUs.

AlexNet Simplified Diagram:

Input (227x227) → CONV (ReLU) → POOL → CONV (ReLU) → POOL → CONV → CONV → CONV → POOL → FC (Dropout) → FC (Dropout) → Output

Conclusion

Pooling is a simple yet vital component that complements the convolution operation. Together, they form the building blocks of deep CNN architectures. By studying classic models like LeNet-5 and AlexNet, we can see the fundamental principles of stacking these layers to build powerful image recognition systems.