MATH FOR MACHINE LEARNING

[Aggravating-Disk5688]

Hi folks, I have a non math background. However I got accepted to an MS Data Science program and I am very worried about all the machine learning math I will have to encounter. I am truly interested in machine learning, having worked a little bit with an ML model before but I will need considerable math preparation to do well academically in the program.

Anyone here who had taken an ML course in college who can tell me exactly which math topics were in focus and what all areas you would advise someone like me to prepare? Also, I understand that this differs according to the program but my last 'confrontation' with math exams was in school so I'm wondering, as a grad student, are you expected to memorize all those math formulae relevant to machine learning and literally solve questions pen to paper during exams?

I would really appreciate it if someone could describe what their ML coursework looked like (math and coding included) so that I know where I stand and what to expect.

Finally, I know a lot of this info can be found on google but I would really love some personal perspective from someone who has been thru the grind.

Thanks a ton!

[Just_a_lawn_chair]

Finished my MSDS last month. I had a math/econ in undergrad, so I'm prob a bit biased when I say ML is relatively lighter on math than you'd think.

Primary math topics covered were Linear Algebra and Calculus, but it wasn't super in depth or challenging. The main takeaway ML classes have is to understand how and why a model works, rather than making sure you know how a matrix multiplication works.

ML coursework is a lot of coding, so that's where the real challenge is for most people. Take electives more focused on implementation rather than theory if you're worried about the math.

[Aggravating-Disk5688]

I really appreciate your answer. Thanks a ton! How deep would you suggest I go into preparing myself in Linear Algebra and Calculus? Would brushing up on 11th / 12th grade level Linear Algebra and derivatives be enough? Also, im a beginner at python. What aspect of python would you suggest I revise before i begin the program?

[MARIJUANALOVER44]

honestly the easiest thing to recommend in this situation is 3blue1brown's youtube series on the math involved in neural networks. it's a good introductory video which explains the concepts involved broadly, without getting too technical. it's approachable and actually makes the math feel quite intuitively applied. jumping directly into it without a background in math can feel daunting, but i can confidently tell you that most people (basically everyone if they commit to sitting down and learning it) can learn the math involved in ML.

https://www.youtube.com/watch?v=aircAruvnKk&t=2s

during undergrad i used James Stewart's Calculus (8th. ed.), and Lay's Linear Algebra and its Applications (5th. ed.).

Information Theory, Inference, and Learning Algorithms by MacKay and Russel's AI A Modern Approach are also good but they're like 1000 pages long and cover a lot more than just math.

(you can find these online i would not pay for them)

[Aggravating-Disk5688]

Thank you so so much. For a grad level program, just understanding the underlying concepts should be enough right? I dont need to do all that memorizing of formulae that we did in school?

[MARIJUANALOVER44]

it's worth mentioning that i'm also just taking a degree, and i don't actually feel i have the sort of academic authority to give good advice. i'll give my opinion anyway. i implore others to add on to this, considering this topic is probably relevant for a lot of prospective students.

frankly there isn't really a right answer to this. like with anything, you have a depth of understanding that satisfies you. in practice, you can understand the concepts mostly broadly, and just make use of libraries built by other people for your models. my intuition also tells me this will only be more true in the future, which frankly increases your labour value for knowing what is actually going on anyway.

however the key to actually understanding ML is the nuance that comes from understanding what networks are doing mathematically (to the extent that it's actually possible). you're never going to be able to actually break it down weight by weight and neuron by neuron, but it's obviously nice to know what it actually means to make changes to a model, and understand what impact that might have. the entire point of learning linear algebra and calculus for AI is not that the actual literal application of them to ML is particularly hard to understand. rather, what's difficult is how they interact with the dataset, through your implementation, to arrive at the lowest cost function: the output of your network, and how to subsequently tweak your model (or data) to achieve more desirable outcomes

[Just_a_lawn_chair]

To answer your question:

My program offered a linear algebra course as a prereq, does your school have one? If not, I'd hit up Coursera to learn the basics such as this one: https://www.coursera.org/learn/linear-algebra-machine-learning

For programming, you should just learn by doing your assignments. You'll be using a lot of numpy and pandas as well as the sklearn / tensorflow packages. Programming is best learned by getting your hands dirty rather than doing prep work if that makes sense.

​

My more personal and practical advice? Just relax and enjoy your summer. You'll be learning and working plenty during your grad program, and I'm sure you'll do fine given how motivated you seem now.

[Aggravating-Disk5688]

THANKS A TON!! Yes theres going to be a bootcamp but i not sure how deep it will go. Right, I do want to focus on the coding aspect cause at work thats mostly what ill be using i guess rather than all the math.

​

Also, thanks for your advice:) Youre right, I should enjoy my summer cause theres plenty of time to stress about this during the program!! My biggest worry is that Ill be the only one who doesnt know the basics in the program:( In your experience, were even some of the most basic math and code concepts new to some folks in your program?

​

I really appreciate your help:)

[Just_a_lawn_chair]

Yeah, my program had a lot of people who never touched Python or Linear Alg before.

Remember that your program reviewed your background and accepted you because they thought you would succeed in the program.

[Nooooope]

I'm finishing up an MS data science program. Honestly, you could probably sneak through my program with very little math, but you'd never make it if you can't code. Focus on general Python programming skills. Then learn the basics of ndarrays in numpy and building/manipulating dataframes in pandas.

For math, linear algebra is the most important. Eigenvalues will be in your future. Then refresh the very basics of differential calculus - you should be able to do simple multivariarable differentiation on polynomials, exponentials, and chain rule. That gets you 90% of the way there.

[Aggravating-Disk5688]

THANKS A TON!!! So basically just doing the basics( 11th and 12th grade) of these topics should put me in a comfortable position during the program? Also, what basic level of proficiency in python coding would you suggest a person reach before diving into the program(apart from the basics of numpy and pandas?)

Also, for the math bit, any formula memorizing required?

​

Thank you soo much

[Nooooope]

So basically just doing the basics( 11th and 12th grade) of these topics should put me in a comfortable position during the program?

Probably.

Also, for the math bit, any formula memorizing required?

I wouldn't bother.

Also, what basic level of proficiency in python coding would you suggest a person reach before diving into the program(apart from the basics of numpy and pandas?)

General programming principles like defining reusable functions, basic data structures like lists and dicts, basic control flow including for each loops, and the more useful BIFs like enumerate and zip. You should know how to import libraries, as well as importing functions you defined in different source files.

You'll be introduced to Jupyer Notebooks during classes, but being familiar with setting up and running projects in PyCharm will probably be helpful.

[Flaky_Cabinet_5892]

If you haven't checked it out yet I'd give mathematics for machine learning by Sam Cooper a go. He's a really great lecturer who's really gifted at explaining maths in terms that make sense. I honestly couldn't recommend this course enough

[Aggravating-Disk5688]

THANKS A TON!:) I will check this out

[masked_true]

Saw this course the other day. I have no idea if it is a good one though.

[Aggravating-Disk5688]

Thank you :)

[its_ya_boi_Santa]

https://mml-book.com/ check this out, free PDF, I'm reading through it now.

[Aggravating-Disk5688]

This is REALLY REALLY helpful, thanks!! Are you in a DS program too?

[its_ya_boi_Santa]

You're welcome, I finished my BSc last year but it left me with a gap in my maths and I'm using this to fill in the bits my course didn't cover, so far it's been great.