# CNN Overview

## Introduction

[Read this: Convolution Neural Network by cs231n](https://cs231n.github.io/convolutional-networks/)

* [image from here](https://developersbreach.com/convolution-neural-network-deep-learning/)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-5ee53b7cf985cf00e0b93fe4fa5b039d144f82a8%2Fimage%20\(225\).png?alt=media)

Example of simple CNN architecture

![VGG-19](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-49091783c7f2589116be286cb247d4ed62df72ee%2Fimage%20\(235\).png?alt=media)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-fa86476c1cfc7bc38d4cf74f532c85acc763f226%2Fimage%20\(224\).png?alt=media)

LeNet-5 (1998): [image ](https://towardsdatascience.com/illustrated-10-cnn-architectures-95d78ace614d)[by Raimi Karim](https://towardsdatascience.com/illustrated-10-cnn-architectures-95d78ace614d)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-4ba69d930fd03e53c33048f0f9695b6911bbbf83%2Fimage%20\(230\).png?alt=media)

AlexNet(2012)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-9d6040c252c96f70fd60350452309b5e520da25a%2Fimage%20\(226\).png?alt=media)

VGG-16(2014)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-3e3a64ed98f5198b50cbf97edbef249df249e476%2Fimage%20\(238\)%20\(1\)%20\(1\).png?alt=media)

## Convolution

## Activation Function

## Pooling

A problem with the output feature maps is that they are sensitive to the location of the features in the input. One approach to address this sensitivity is to down sample the feature maps. This has the effect of making the resulting down sampled feature maps more robust to changes in the position of the feature in the image, referred to by the technical phrase “_local translation invariance_.”

[Read this: Introduction to Pooling Layers by Machine Learning Mastery](https://machinelearningmastery.com/pooling-layers-for-convolutional-neural-networks/)

## Fully Connected Layer

## LeNet-5

[Read here](https://towardsdatascience.com/illustrated-10-cnn-architectures-95d78ace614d)

LeNet-5 is one of the simplest architectures. It has 2 convolutional and 3 fully-connected layers (hence “5” — it is very common for the names of neural networks to be derived from the number of _convolutional_ and _fully connected_ layers that they have). The average-pooling layer as we know it now was called a _sub-sampling layer_ and it had trainable weights (which isn’t the current practice of designing CNNs nowadays). This architecture has about **60,000 parameters**.

**⭐️What’s novel?**

This architecture has become the standard ‘template’: stacking convolutions with activation function, and pooling layers, and ending the network with one or more fully-connected layers.

**📝Publication**

* Paper: [Gradient-Based Learning Applied to Document Recognition](http://yann.lecun.com/exdb/publis/index.html#lecun-98)
* Authors: Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner
* Published in: \_\*\*\_Proceedings of the IEEE (1998)
