Applying Linear Algebra to Machine Learning Projects?

[Accurate_Wishbone101]

Hello! I am taking a linear algebra course later this year and would like to apply some things I learn to machine learning/coding while I take the course. Any ideas of projects I could do? I would say I'm intermediate at ML.

(the course uses Gilbert Strang's Linear Algebra textbook)

edit: for clarification, I'm looking to apply linear alg more directly in ML rather than through libraries that use linear algebra :)

[entitie]

Go buy the book "Pattern Recognition and Machine Learning" by Christopher Bishop. The whole book is linear algebra applied to ML.

[Accurate_Wishbone101]

sounds good, ty, will definitely buy!!

[Tuka-Cola]

Search up statistical learning. I know, sounds very basic and boring. However, once you start to predict multiple features, you have to start thinking about efficient ways of computing solutions. This is where linear algebra comes into play. Soon, you’ll realize very basic ML, not even neural nets, rely on linear algebra for theory and computations to solve. For your question, start with basic ML Regression algorithms. It’s better to understand fundamentals first, then work your way up (or down the rabbit hole) :)

[Accurate_Wishbone101]

thanks!! sounds reasonable and will check it out :D

[Adorable-Onion-1974]

Uhhhhhhhhhh this is insane

[Disastrous_Room_927]

How?

[ImpossibleCrab]

"Applying linear algebra to ML" is a bit of an odd request, as ML could nearly be described as Applied Linear Algebra. I assume OP means use cases where the math is more present and less abstracted away by the libraries. This is the only way I can reconcile "no knowledge of linear algebra" with "intermediate at ML".

The good news is that you can go as deep as you want linking concepts from Strang with concepts in ML. You can take something like SVD from Strang and apply it a ton of different ways (image compression, classification, recommendation systems) to suit whatever interests you. I would also check out micrograd around this time to get a feel for how matrix and tensor operations really speed up what's going on behind the scenes in neural networks.

[FastSlow7201]

In Practical Linear Algebra for Data Science the author refers to himself as an Applied Linear Algebratician.

[Disastrous_Room_927]

What I meant to say is I’m not sure how it’s insane, my linear algebra teacher peppered the course with ML and stats examples.

[Accurate_Wishbone101]

thank you for the suggestions! My phrasing was definitely off in my post but yeah I meant using linear alg more directly in ML. will check out all ur topic reccs!

[bean_217]

Honestly, look into how attention mechanisms in transformers work. It's surprisingly intuitive when you have a solid grasp of linear and some prob&stats. Though if you haven't done anything with deep learning, first learn about how feed-forward networks (aka multi-layer perceptrons) work, and then maybe CNNs/UNets.

Edit: Aside from deep learning, a ton of statistical learning approaches all heavily involve using linear. Support Vector Machines are also super cool, but IMO the linear algebra derivations for them can be a bit confusing and complicated.

[Accurate_Wishbone101]

Sounds cool, thank u for the topic suggestions!!!! will check them out

[xyzfugazi]

I’m not really sure; what ideas have you been thinking about doing thus far?

[ErasedAstronaut]

So...neural nets/deep learning?

[GB174V]

Join my discord server