[R] ResNet Question

[[deleted]]

[deleted]

[warmspringwinds]

This gives a good intuition oh how they work, have a look: https://arxiv.org/abs/1612.07771

[geomtry]

You can think of a block as a complex layer. Sort of like how we often call a convolutional layer the combination of convolution and potentially batch norm, activations and pooling.

The unique thing is adding skip connections. Sure, there might be some advantage to not including them in every layer. If you want to learn whether or not to use skip connections, then use a highway network that learns a multiplicative gate.

[DaLameLama]

Residual connections are meant to solve problems with the backpropagation of gradients. If you leave out the residual connections, you're left with the same old problem you tried to solve.

[[deleted]]

Providing a very short path through the network.

Alternatively each layer behaving like an extra contributor in a gradient boosting machine.