ResNetv2 snippet

The ResNetv2 is a neural network architecture used for image classification, regression and feature extraction. It uses residual learning and a skip connection to add forwarded activations to the output activation-map.

Deeper neural networks have historically been hard to train. Residual learning with skip connections made it possible to successfully train deeper models than ever before, there are well-performing networks with over 1000 layers. For most recent models we now observe that deeper models are more powerful.

The idea is to build a network consisting of branches with skip connections. For each branch, you then learn the difference, the residual activation-map, between the input and the output of the branch. This residual activation-map is added together with the previous activation-maps building the “collective knowledge” of the ResNet.

ResNet branch with skip connection
Figure 1. ResNet branch with skip connection

ResNetv2 architecture

There are many variations of the ResNet v2 architecture. We define the ResNet v2 architecture as follows:

  • Each convolution is preceded by batch normalization and an activation as in the second paper Identity Mappings in Deep Residual Networks.

  • The architecture consists of Bottleneck blocks from the original ResNet paper (Deep Residual Learning for Image Recognition)

Two versions of ResNet v2 is implemented on the platform:

  • Small for images close to 32x32 pixels.

  • Large for images 224x224 pixels and larger.

How to use the ResNet snippet

To add a ResNet snippet open the Snippet section in the Inspector and click on one of the ResNet snippets.

When using the ResNet snippet you could consider the following things:

  • In the Input block, we recommend that you use image augmentation.

  • Change the number of units in the last Dense block to match the number of classes you have. Also, change the activation.

  • Set the correct Loss function in the Target block and specifying the correct loss.

  • For optimizer we recommend you to use ADAM with learning rate 0.001 or SGD with momentum and learning rate 0.01.

References

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun: Identity Mappings in Deep Residual Networks, 2016.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun: Deep Residual Learning for Image Recognition, 2015.

Try the platform