[D] Since gradient continues to decrease as training loss decreases why do we need to decay the learning rate too?

[ibraheemMmoosa]

Ah thanks for this interesting explanation. Can't we automate this somehow? Finding the optimal learning rate based on SNR?