Residual Networks (ResNets) are a type of deep neural network architecture that was introduced by Kaiming He et al. in 2016. ResNets are designed to help address the problem of vanishing gradients, which occurs when the gradients of the activation function in a deep neural network become very small or zero, making it difficult for the network to learn and converge.
The main idea behind ResNets is to introduce skip connections or shortcuts that allow the gradients to flow more easily through the network. In a standard neural network, the input flows through a series of layers, and the output of each layer is fed as input to the next layer. In contrast, in a ResNet, the input is added to the output of the layer, which creates a "residual" connection.
This residual connection enables the network to learn a residual mapping, which is the difference between the input and output of the layer. By learning this residual mapping, the network can focus on learning the more complex features of the data, rather than trying to learn the identity mapping. This can lead to faster convergence and improved performance, especially in deep neural networks.
The ResNet architecture consists of several residual blocks, each of which contains several convolutional layers and a skip connection. The skip connection allows the output of the residual block to be added to the input of the block, which enables the gradient to flow freely through the block.
One of the most popular ResNet architectures is ResNet-50, which has 50 layers and uses bottleneck building blocks, which consist of three convolutional layers: 1x1, 3x3, and 1x1 filters. This design helps to reduce the number of parameters while increasing the depth of the network.
ResNets have been shown to be very effective in various computer vision tasks such as image classification, object detection, and segmentation. For instance, ResNets have won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) several times since their introduction.
In summary, ResNets are designed to help address the problem of vanishing gradients in deep neural networks. By creating residual connections between layers, ResNets enable the gradients to flow more easily through the network, which can lead to faster convergence and improved performance.