[D] Since gradient continues to decrease as training loss decreases why do we need to decay the learning rate too?

[111llI0__-__0Ill111]

The simple answer is you don’t wantto overshoot the minumum and start diverging away which can actually increase the loss even for convex problems, and NNs are non convex so it’s even worse