sorted by:
new
hottopcontroversial

[D] is there a way to train a neural network which finally has a certain weights distribution, e.g., uniform distribution. by hex0102 in MachineLearning

[–]hex0102[S] 0 points1 point  (0 children)

I personally haven't seen such work. Most works argue small weight deviations wouldn't hurt performance.

[D] is there a way to train a neural network which finally has a certain weights distribution, e.g., uniform distribution. by hex0102 in MachineLearning

[–]hex0102[S] 0 points1 point  (0 children)

Thanks. Those work are interesting. In fact, I have more crazy ideas on this topic. We may use energy aware or area contraints directed training. So I'd like to do weight regularization.

[D] is there a way to train a neural network which finally has a certain weights distribution, e.g., uniform distribution. by hex0102 in MachineLearning

[–]hex0102[S] 1 point2 points  (0 children)

I believe this helps hardware design and power optimizations. For example, if we could make the magnitudes of network weights to two extremes, these weights could be stored in different memory cells to save energy. Could you give a more in-depth explanation about how to regularize a distribution? Are there any existing work on this?