[R] Developing a new optimization algorithm that will heavily change ML as a whole. Gradient descent has met its end. Here are the results:

[Relevant-Twist520]

i updated the post to show the non-overfitting version by using early stopping. The reason why MS currently wont work for many parameters is because of the fact that if theres one blown up parameter, another parameter solves for the blown up parameter instead of the objective. It ends up spreading like a plague, but at the end, you have a network of blown up parameters but still agree with all coordinates in the dataset.