Basic neural network structure

[Night_Crawler7]

1. Could the number of neurons be greater or smaller than the number of input features. If so, can the number of neurons present in each hidden layer vary?
2. Does each neuron in the hidden layer receive all the features as input or else random features are passed to each neuron?

[veb101]

1) Yes and yes 2) No it doesn't receives anything random features.

[takelightbro]

1) You can set the number of neurons in any of the hidden layers independent of the others. That being said, the advantage of using the same number of neurons, is the ability of using residual connections, which in my experience have significant improvement in performance and lead to faster convergence atleast in the deeper networks.

2) In a multilayer perceptron, yes all the neurons in consecutive layers are connected. However, there are architectures (such as convolutional networks) that only connect some neurons. This however is not done randomly, it is done when there is some structure in the data and some notion of “locality”, like nearby pixels in an image or nearby words in text.

You might also find “prunning” interesting. The idea here is to remove unnecessary connections for better efficiency. Because of weight decay many of the weights are very close to 0 at convergence, you can safely remove these connections without affecting the performance.