> For the complete documentation index, see [llms.txt](https://ykkim.gitbook.io/dlip/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ykkim.gitbook.io/dlip/deep-learning-papers/vgg.md).

# VGG

[Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/pdf/1409.1556), Simonyan, Karen, and Andrew Zisserman. 2015 ICLR & preprint arXiv (2014)

![VGG architecture image from https://neurohive.io/en/popular-networks/vgg16/](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-051be1b2f996722d41da4d4c35c03258e8e27e51%2Fimage%20\(247\).png?alt=media)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-f761424d31fe0fbe5895b84cce3c886f106ba398%2Fimage%20\(254\).png?alt=media)

VGGNet is invented by VGG ([Visual Geometry Group](http://www.robots.ox.ac.uk/~vgg/)) from University of Oxford. Though VGGNet is the **1st runner-up**, of **ILSVRC 2014 in the classification task.** Accuracy error less than 10%

> GoogLeNet is the winner of ILSVLC 2014. **VGGNet beats the GoogLeNet and won the localization task in ILSVRC 2014.**
>
> Improved from AlexNet(-8) 16.4% to VGG-16 7.3%)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-0a8cd394f173c0b3292ceb6d5900516994037742%2Fimage%20\(249\).png?alt=media)

A very important paper on CNN. It uses only 3x3 CONV and many networks are based on VGG architecture.

## **The Use of 3×3 Filters**

* Instead of large-size filters (such as 11×11, 7×7) as in AlexNet
* Repeats 3 layers of 3x3 CONV instead of 1 time of 7x7. They both cover 7x7 receptive field
* VGG has fewer parameters, more non-linearity.

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-6707476011cfeff9dac0d01726a7b18d9ff1fd65%2Fimage%20\(253\).png?alt=media)

![a) Feature Map from 7x7 CONV has 7x7 receptive field. b) Feature Map from 3 layers of 3x3 also has 7x7 receptive field. Image from https://medium.com/@msmapark2/](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-d14698817cb79f82994257d44a7ba05215c78c22%2Fimage%20\(245\).png?alt=media)

## **Ablation Study: VGG-16, 19**

1. VGG-16 (Conv1) obtains 9.4% error rate, which means the additional three 1×1 conv layers help the classification accuracy. 1×1 conv actually helps to increase non-linearlity of the decision function. Without changing the dimensions of input and output, 1×1 conv is doing the projection mapping in the same high dimensionality.
2. VGG-16 obtains 8.8% error rate which means the deep learning network is still improving by adding number of layers.
3. VGG-19 obtains 9.0% error rate which means the deep learning network is NOT improving by adding number of layers. Thus, authors stop adding layers.

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-f36baa2615021ededd51a2a714e41e916dc4064c%2Fimage%20\(248\).png?alt=media)

## **Multi-Scale Training and Testing**

Rescale from 224 to 256\~512px. Then crop to 224px which contains the object fully or partially. Has the effect of data augmentation with scaling and translation, which helps to reduce overfitting.

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-3147e5ea85b6c93f8ba9981be2d0938b60e3e396%2Fimage%20\(250\).png?alt=media)

![](https://3883264845-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MR8tEAjhiC8uN1kHR2J%2Fuploads%2Fgit-blob-0f6516640589b94a2ee0a5c236c2c2ebc00e5673%2Fimage%20\(246\).png?alt=media)

## Training

* Dataset: ImageNet of 256x256x3
* Input: 224×224×3 image with data augmentation from multi-scaling
* Batch normalization
* Multinomial logistic regression loss
* Mini-batch GD with momentum
  * Batch size: 256
  * Momentum v: 0.9
  * Weight Decay: 0.0005
  * Learning rate : 0.01 decreased factor of 10