## Dropout technique

Dropout is one of the simplest and the most powerful regularization techniques. It prevents units from complex co-adapting by randomly dropping units from the network. [N. Srivastava et al., 2014] Below table is a list of famous deep networks which use dropout techniques.

## Status of dropout technique usage in famous deep networks

Model Dropout layers Remark
AlexNet [Alex Krizhevsky et al., 2012] Used in two fully-connected layers Won the 2012 ILSVRC (ImageNet Large-Scale Visual Recognition Challenge)
ZFNet [Matthew D. Zeiler et al., 2013] Used in two fully-connected layers Won the 2013 ILSVRC
VGG Net [Karen Simonyan et al., 2014] Used in two fully-connected layers Best utilized with simple and deep CNN
GoogLeNet [Christian Szegedy et al., 2015] Used in one fully-connected layer Won the 2014 ILSVRC
Generative Adversarial Networks [Ian J. Goodfellow et al., 2014] Applied in training the discriminator net Various usage such as feature extraction, generating artificial images
Generating Image Descriptions [Adrej Karpathy et al., 2014] Used in all layers except in the recurrent layers Combination of CNNs and RNNs
Spatial Transformer Networks [Max Jaderberg et al., 2015] Used in all layers except the first convolutional layer Introduce of a Spatial Transformer module

## Notices

The list will keep updated

Tags:

Categories:

Updated: