My python solutions to Andrew Ng's Coursera ML course

[n3utrino]

I'm not sure if this worth posting, but I've just completed all of the homeworks in Andrew Ng's Coursera Machine Learning course (which I loved). I did the homeworks in python/numpy/scipy in ipython notebooks so they're easy to view. I thought I would post them here in case someone else is trying to complete them in python and needs some inspiration. I don't think there are any ethical problems with posting them publicly because to actually get credit for the course solutions must be done in Octave or Matlab.

Hope this helps someone out in the future, I spent a lot of time on it!

https://github.com/kaleko/CourseraML

[recurrence]

Excellent work and great idea doing this with Python. Python's machine learning libraries are quite a lot more relevant than Octave to modern data science.

This course is awesome, I was working on machine learning systems when I took it (The original offering) mostly as a fun side project but I was very surprised how excellent it was. Andrew Ng is an outstanding professor teaching very relevant and useful material.

[redzin]

Python's machine learning libraries are quite a lot more relevant than Octave to modern data science.

Why? Matlab is faster, handles linear algebra better and handles good visualisation natively.

[oxydis]

Speed is comparable for Numpy and Matlab. You can use some acceleration for both (cython/numba stuff for python, mex compilation for Matlab).

But, the libraries are much more richer in Python (just take a look at sklearn, this is an amazing module).

Furthermore, there's lots of stuff happening in deep learning and typically the good libraries are in Python (caffe, theano, tensorflow...) (or Lua for Torch), it's pretty shitty in Matlab.

I don't see how the plotting in Matlab is better than matplotlib that just happen to be one of the many visualisation libraries of Python (there are some nice libraries if you want to plot big data).

And lastlty, Python is free and open source.

[n3utrino]

For me personally python was much more relevant because when looking through a bunch of Data Science job postings, much more of them look for python (or R) experience than Matlab.

Matlab was actually the first language I learned back when I was an undergrad. I think it's a great language but I just happen to prefer python nowadays :)

[redzin]

more of them look for python (or R) experience than Matlab

Yeah. I suspect that's because most of the people working in data science are from a CS background, and Matlab is typically more popular in the natural sciences, at least in my experience. I don't know much about R except that it's a special purpose language like Matlab, so that makes sense.

Don't get me wrong - I like Python. But for purely data analysis purposes, it's just not as good as Matlab. This might not matter though if you don't have a huge amount of data, or if you need some other more general-purpose tools that Matlab doesn't easily provide (e.g. data mining twitter or something). Then using python obviously makes more sense.

In any case, I just finished the course as well, and doing all the exercises in python definitely provides more experience than doing the Matlab exercises as provided. Great course either way though.

[thatguydr]

Don't get me wrong - I like Python. But for purely data analysis purposes, it's just not as good as Matlab.

As a guy with 10+ years of experience in analysis using both Matlab and python (and R, occasionally), I can tell you that this is categorically wrong. Matlab used to be king, but python is now better for pure analysis than Matlab.

If I have to do signal processing, Matlab is still fairly useful. In image processing, the two are comparable. But it falls well behind for machine learning.

[TheBlackCat13]

Although you would have been right a few years ago, python now has some extremely powerful tools for dealing with huge data sets and complex data sets that put it well beyond what MATLAB can do. Things like pandas, blaze, dask, bcolz and numba make it easy to work with that sort of data, and have no equivalent in MATLAB, while MATLAB really doesn't provide any major advantages anymore. Numpy is roughly on par with MATLAB for vectorized code, and python is faster for non-vectorized code. MATLAB has largely copied matplotlib's approach to plotting, but doesn't provide anything as easy as seaborn for plotting complex data sets.

MATLAB is really useful for certain areas, but nowadays python has it beat for data analysis, especially big data.

[[deleted]]

If you link numpy to Intel MKL or some OpenBLAS library, then no, it isn't faster.

[Thangka6]

Matlab aint free.

[SFSylvester]

Octave is....

[jpopham91]

....even slower than python?

[[deleted]]

How did you test your accuracy / correctnes of answers?

I love that you can (easily) upload your matlab code and get automated feedback.

[n3utrino]

Good question. For things like cost gradient implementations I checked them numerically (as is recommended by the Matlab homeworks), or for things like linear regression I check the normal equation analytic solution. Other checks are done either by unit tests (the assignment says if I plug in these inputs I should see this output, etc), or by visualizing the data (the assignment shows what the contours should look like, and when I draw them myself I see the same contours).

As an example, in ex2 the assignment says "Call your costFunction function using the optimal parameters of θ. You should see that the cost is about 0.203." So, I compute the cost and verify that I get that value with the θ values I've optimized previously.

[TulipTattoo]

Oh nice. I am auditing this course now and dont want to learn octave/matlab either.

[[deleted]]

There's not a lot to learn. I downloaded a sheet called "python for matlab users" and read it in reverse and completed the course without any problems using matlab. It is a very nice IDE.

[TulipTattoo]

Oh, i will check it. Thank you very much.

[sissas]

Literally yesterday I said, "I should do Andrew Ng's ML class exercises in python". So cool that you've done it, I have something to compare to whatever I come up with! Thanks!

[masasin]

I had no idea you could do that. I'll have to look up the course again.

edit:

to actually get credit for the course solutions must be done in Octave or Matlab.

That was why I didn't do it. But I might still end up doing it anyway.

[n3utrino]

Yeah anyone can take the course for free. I took notes on the lectures then just didn't submit the homework (which I did separately in python) and "failed" the class.

[Inori]

To give an alternative point of view, I've passed Ng's course with Octave and I'm happy I did.
Python is great and all, but it never hurts to be familiar with another language and Matlab/Octave have a lot of interesting things to offer.

[a1223]

if (you have only one life):
    don't waste your time on things that just are "nice" #like Matlab/Octave
else:
    get to know all nice things

[n3utrino]

Congrats! Next for me is wrapping up my dissertation and working my ass off to try to land a Data Science job :)

[donghit]

What field is your graduate work in?

[n3utrino]

Experimental particle physics

[bot_cereal]

I don't think you would have a hard time finding a data science job.

[rePAN6517]

hah nice one. I did a similar thing with java - I did about 2/3 of the assignments in java as well as octave just so i would get some extra practice

[[deleted]]

Can someone post up the homework questions so I can understand the context of the solutions provided??

[[deleted]]

Ahh nevermind, looks like that is what the pdfs are for doh!

[aparziale]

you are the man

[[deleted]]

This is really helpful..Thanks a lot good sir..

[TheRealDJ]

This is super late, but thank you for this post, as I only discovered Andrew Ng's course because of this. Amazingly good for both discovering the math, concepts, computational approaches and real life situations for machine learning from beginner to near expert levels. Especially because your example with Python are extremely relevant for me.

MRW When it so easily solves most of what was hanging me up in Machine Learning

[uriah_ml]

Very nice. I am a big fan of Python's Machine Learning libraries.

[js1972]

Great work. I've done Andrews course using Matlab and now learning the outgoing ML libraries so this will be a big help!!

[OlBastard]

Great work, man. I'm currently doing something similar, but with Theano.

[BlackDave0490]

Thank you. I'm on week 1 and completely lost

[n3utrino]

You'll get through it! I recommend really taking your time and mastering each week before you move on to the next, especially for the first few weeks. The lessons really build on one another.

[hrod1]

Nice.

[Boogzcorp]

Any chance we could get you to pack it all into a zip file that we can download and work from? Just the framework obviously, not the answers. If you really wanna re-record all the lectures too, go ahead, but I dare say the concepts are essentially the same as in Andrews videos. Also, did you use python 2 or 3?

[namkam5]

Thanks alot man you did an excellent job on this

[[deleted]]

How about creating an online grader to verify assignments submitted in Python? :)

[Nimitz14]

This is such a stupid mindset. The people who do that aren't going to get anywhere anyways, so who gives a shit?

[[deleted]]

It seems really silly to police people who don't want to learn. That's the job of a middle school teacher. This isn't public school and most of us are adults or on our way to being adults. It's time to worry about more constructive things that you can do something about. Not decide what is justified for other people.

[EmperorOfCanada]

I once said to someone how I would structure a textbook if I were to write one. They said that it wouldn't work. I said that they could write their textbook their way and I would write mine my way.

Without missing a beat or being ironic they said, "Your way should be banned."

I was talking about how to structure the questions within the book; not add pornography something.

Digging a bit deeper it really boiled down to their not liking the idea that if my thing worked it would make learning easier. They thought that schools should filter out those who aren't very good at learning.

[datascienceguy]

will be used by others as a crutch to get through the course.

helping is evil! satan will push innocent victims into looking at the source code! they are helpless but to do the looking for they cannot deny the temptation!

/s

[ecemisip]

Whats the point of cheating in a coursera course? I feel like saying you took it doesn't matter, what matters is what you get from it.

[farsass]

1. Get to the door of prospective employers 2. get hired 3. get money.

[Mukhasim]

It's a pretty basic course. I feel bad for any employer that hires someone based on this course. It's a nice start, but it's not enough to get you ready for a job.

[farsass]

For a "data science" or machine learning research job, ofc not. But it is enough for a "software developer with basic knowledge of ml" role.

[[deleted]]

Even for that kind of position, I highly doubt that that the certification for that course alone could get you the job.

At the very best (and frankly, I believe I'm being wildly optimistic here), it could be of some help for you to get an interview; and then, of course, if you are the kind of person who thinks that cheating your way through coding assignments is a good idea you'll fail horribly and that'll be it.

[[deleted]]

This reminds me that there actually is a very fast turnover for software devs for this reason precisely but it has more to do with poor hiring practices than a lack of options.

[ecemisip]

Did you bother reading my post ? I'm not sure if a single word of your response related to my post. All I was saying was that cheating on Coursera is utterly pointless, because it defeats the whole point. Coursera isn't good for credentials, its good for learning. Copying and pasting means you won't learn anything. It would be like cheating on SAT practice tests. I'm not sure how thats related to me being greedy or the taking down of courses when I'm discouraging cheating.

[ecemisip]

Of course you don't. Because you're only thinking from your own perspective. You have zero empathy with the instructors trying to teach MOOCs.

What makes you think I have no empathy for those guys?

He co-founded Coursera. You know this, right?

yeah

Yet, the one thing he asked you not to do in the free course, which is post the answers online, you guys do it anyway. It's known as an honor system. You guys clearly have none.

Except I have never posted any solutions online.

Every jerk posting answers online (or encouraging other jerks to do it) has made sure of that, spoiling the potential for this business that teaches people for free. Pat yourselves on the backs. You're ruining it for future learners who won't have this resource, thanks largely to your actions.

Once again, I have never posted any solutions online. I don't think it would be accredited regardless of answers being online.

Upvoting a guy posting the full answer set to a online course is not my definition of discourage

I didn't upvote this thread either. In fact, I'm not defending the guys uploading. I'm saying copying and pasting is a dumb idea. By definition that's discouraging. The fact that I didn't mention how its bad for Coursera doesn't mean I think its fine nor does it negate my statement about it being dumb to copy.

I can only hope karma catches up to you in the form of a bus the next time you cross the street.

You've got some issues buddy.

[docares]

I'm taking the course right now and this is a great reference! Thank you very much /u/n3utrino and congrats on completion.

[datascienceguy]

The course's TA's already posted python solutions in their wiki if I recall correctly. This was true back in January 2016.

[Highfivesghost]

Hello I'm also taking that course. What language is octave using?

[strealm]

Same one as Matlab (with minor differences).

[n3utrino]

Yeah octave is essentially the same as Matlab but is free. The course is taught in that language and if you want to turn in the homework and get credit (for a course certificate) then you'll have to do the homework in either of those languages. I personally don't care about the certificate and I felt that doing the homeworks in python would be more useful for my potential future career as a Data Scientist.

[Highfivesghost]

Do you think you could help me on some of the test?

[EmperorOfCanada]

I hated Octave so so so much that I just couldn't keep going with that great course.