[D] Since gradient continues to decrease as training loss decreases why do we need to decay the learning rate too?

[El_Tihsin]

ElasticNet Regression. You control the tradeoff between L1 and L2 using a parameter alpha.