CNN Overview

Introduction

Read this: Convolution Neural Network by cs231n

• image from here

Example of simple CNN architecture

VGG-19

LeNet-5 (1998): image by Raimi Karim

AlexNet(2012)

VGG-16(2014)

Convolution

Activation Function

Pooling

A problem with the output feature maps is that they are sensitive to the location of the features in the input. One approach to address this sensitivity is to down sample the feature maps. This has the effect of making the resulting down sampled feature maps more robust to changes in the position of the feature in the image, referred to by the technical phrase “local translation invariance.”

Read this: Introduction to Pooling Layers by Machine Learning Mastery

Fully Connected Layer

LeNet-5

Read here

LeNet-5 is one of the simplest architectures. It has 2 convolutional and 3 fully-connected layers (hence “5” — it is very common for the names of neural networks to be derived from the number of convolutional and fully connected layers that they have). The average-pooling layer as we know it now was called a sub-sampling layer and it had trainable weights (which isn’t the current practice of designing CNNs nowadays). This architecture has about 60,000 parameters.

⭐️What’s novel?

This architecture has become the standard ‘template’: stacking convolutions with activation function, and pooling layers, and ending the network with one or more fully-connected layers.

📝Publication

• Paper: Gradient-Based Learning Applied to Document Recognition

• Authors: Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner

• Published in: _**_Proceedings of the IEEE (1998)