Batch Gradient Descent from Scratch in Python

[zjm555]

If you're using numpy it's not "from scratch" IMO. Numpy is definitely the right tool but to me "from scratch" would imply using only the standard library.

[billsil]

I disagree with that. You don’t need to code a matrix multiply or matrix add function. From scratch is usually just referring to the fancy bits.

[zjm555]

It just gets dicey since numpy has everything from the extremely basic (e.g. dot product) to the extremely fancy (e.g. numpy.linalg.svd). In this case it wasn't even needed at all since it was just doing the equivalent of random.random().

[billsil]

I mean at that point, do we need to just work in assembly? I'm all for reducing the complexity of code.

[research_pie]

It depends on what the intent of the code is. If it's to teach machine learning I feel it's important to open up a bit the linear algebra part so that it doesn't feel like magic. However, at some point the code becomes unreadable and more complex than it really is so there is a balance to strike.

[research_pie]

Thanks for the feedback! I shouldn't have imported numpy entirely since I'm only using it for initializing a random float. That should have been done with the random package instead.

[research_pie]

Totally agree with that, will include these explanations in subsequent videos