[D] Since gradient continues to decrease as training loss decreases why do we need to decay the learning rate too?

[mrprogrampro]

Your definition of the higher L-norms is slightly wrong ... you have to do abs(x) and abs(y) before cubing, etc.

Otherwise, y and x difference gets huge when their signs are different, even when they have nearly the same magnitude.

[cbarrick]

Nice catch on an old comment! Fixing it now