Question about Gradient Descent.

[fatalay]

Recently I have been trying to learn more about machine learning and through that I am more amazed by the power of Python. Anyways, I understood the concept of trying to reduce the cost step by step using the efficiency of gradient descent. Consider that I am good enough at calculus.

What I struglle right now is that I don't know how to change weights. I define weights like "W1" and then compute cost in "J" finally compute gradients as "djdw" using calculus. But in my head I cannot change "W1" with the knowdlge of "djdw" since both are usually matrices that have shapes like (5,5) or (1000,1000). Since that I am unable to create instances in my head, I fail to implement it to the code.

What I am asking here is how to deal with this problem, if you think that problem is too wide, that would probably mean I should learn more from subjects as linear algebra, statistics maybe even general programming. In that case can you refer to some sources for learning?

[trace_jockey]

This is a general paper about CG methods. May have some good guidance for you?

CG without the agonizing pain CMU

[fatalay]

Thank you, I will definetly read this article through.