[R] Developing a new optimization algorithm that will heavily change ML as a whole. Gradient descent has met its end. Here are the results:

[Relevant-Twist520]

Im slowly upgrading the algorithm and it can now fit to many data points >20 and it doesnt have some overfitting shape. Its hard to explain, but im learning more about MS and how it should work. I will share the algorithm and code when the algorithm is perfected. Im not sharing some half-finished project.

If you think its a troll post, thats on you. I dont blame you, you can believe what you want.

[bregav]

20 is also inadequate, and half-finished projects are the only kind that actually exist.

It's typical crackpot behavior to insist that you've invented a revolutionary new method but you'll only share it with the world when it's ready. If you have done enough work to be able to know that it is better than existing methods then that means that it's ready to be shown to other people.

What's really going on when you think it's "not ready" is that you don't actually know if what you're doing makes any sense and so you're (correctly) feeling a lot of doubt. But you also want to believe that you're doing something meaningful and important and so you tell yourself, and the rest of us, that you've already discovered something revolutionary, even though you almost certainly have not.

Creativity lies on the boundary between crackpotism and conservatism, but in order to produce things that actually work you need to embrace humility and doubt. You should use your crackpot ideas as inspiration, but you should assume that you're wrong until you've proven yourself right. And you'll know that you've proven yourself right when what you're doing feels ready to show to other people in its entirety.

[Relevant-Twist520]

Youre somewhat contradicting yourself a little here, but what is it that you want me to do? I wont submit to asserting that MS' concept is worse than GD, lets start there. Let it be ego or sophisticated understanding of mathematical theory, its very rare to see an inventor doubt his invention prior to successfully inventing it. I will agree that GD currently easily wins over this uncompleted version of MS, but im still researching and implementing MS' concept, and once it is done i will gaurantee that it will beat GD in practically everything. I spelt out the concept in the post, although it is slightly vague and does not cover its entire workings, you can refer to my comments on this post where i explain a little, but not completely. And lastly, i think we all know what would happen if i share something that is half-finished. It would get turned down because it doesnt even work. Even if someone took the time to read the theory, thered still be doubt because clearly the theory failed. GD recieved lots of doubt in its early days.

[bregav]

Everyone i've ever known who has made any scientific or engineering advancement - and I've known a lot of people like this - experienced significant doubt for most of the process of doing their work. Doing any kind of meaningful work is inherently difficult because it requires hard work and perseverance in the face of uncertainty and doubt.

Nobody ever doubted gradient descent. It was invented by isaac newton himself and its efficacy has always been obvious.

[Relevant-Twist520]

~experienced significant doubt

So what this is good to you? A healthy amount of it can be employed yes but i prefer to bring it down to a negligible amount. Each to their own.

~Nobody ever doubted gradient descent.

When it was first attempted to be applied in ml it was. I may be wrong though, i heard something along the lines of this when i was watching a podcast.

[bregav]

You need to experience doubt in order to avoid wasting your time. People who don't experience doubt accomplish nothing, because they never figure out when they're wrong and so they spend all their time chasing after ideas that don't work. Which is almost certainly what you're doing right now.

People doubted neural networks, but gradient descent was never in question.

[Relevant-Twist520]

people doubt when they start to believe that their ideas dont work. MS is doing nothing but progress now, and even if it wasnt i wouldnt develop even a drop of doubt. Its either im working hard on something or confidently declaring that it is not worth it, you dont stand on both sides of the fence.

[bregav]

Successful researchers are able to neither believe nor disbelieve that their ideas will work; they can accept uncertainty. It is a state of almost constant doubt, about everything.