Add

This block can take between 2 and 5 inputs, and returns a single tensor containing the element-wise sum over all inputs.

All the inputs must have the same number of dimensions.

Along each dimension, an input can only have the same size as the other inputs, or a size of 1. If an input has a size of 1 in a given dimension, its existing values are automatically duplicated until the size in that dimension matches the size of the other input, similarly to numpy’s broadcasting.

For example, you can add inputs of shapes:

  • 16x16 and 16x16

  • 1x1x3 and 32x32x3

  • 64x1x64 and 64x100x64

But you can’t add inputs of shapes:

  • 16x16 and 20X16

  • 10x1x3 and 32x32x3

  • 20x20 and 20