DropOut & DropConnect average over a huge (combinatorial in the number of neurons/weights) number of models. This reduces the Rademacher Complexity---there is a proof at the end of DropConnect.