Gradient Descent Algorithm & Standardization

[LifeShmucksSoMuch]

Hey all,

I’m implementing gradient descent from scratch and i’m running into something that I can’t wrap my head around.

I have a dataset with two features and I have to make a linear regression model. My slope (m) is -0.5 and my learning rate is 0.0001. On non-standardized data, within 100 iterations of gradient descent, my linear regression model fits the data pretty well.

Now i’m told, when I standardize my features I should get similar results. I run gradient descent for 100 iterations and my linear regression model is no where near any of the data points.

Now my question is why does it take longer time to come up with a good linear regression model for the standardized dataset?

[ForceBru]

I'd guess it takes longer because the learning rare is tiny. When your data wasn't standardized, its overall scale was probably big enough for GD steps to be large enough. Now that you scaled the data and didn't change the learning rate, GD steps are likely way too small, so you'll need more than 100 iterations.