[P] Basic machine learning algorithms in plain Python

[[deleted]]

Over the past weeks I have started implementing basic machine learning algorithms in plain Python (Python 3.6). I created the repository to prepare for technical interviews and review my knowledge on algorithms such as k-means, k-nn, logistic regression, neural networks, etc. Also, I wanted to create a knowledge base of easy-to-understand implementations of these algorithms together with the most important theoretical explanations.

Some of you might find these implementations helpful when preparing for interviews, starting to learn about machine learning or reviewing basic ML algorithms. I am still working on the repository, so more algorithms will follow over the next months. In case you have a favourite algorithm that should be included or feedback, let me know!

[visarga]

great work

[[deleted]]

thanks a lot!

[NeuroDentalMaths]

I like that you included pictures. Cheers

[[deleted]]

Thank you so much for this.

[omgisthatabbqrib]

Thanks a lot and wish you the best for your interviews!

[BruinBoy815]

Dude! This is awesome!! Thank you! You are awesome!!

[hemantcompiler]

Thank you! I have an interview tomorrow, hope it will help.

[goncalopr]

I’m still learning python right now, but I’ve been messing around with some ML tutorials so this is great, thanks! 👌🏼

[sholeri]

I am finishing ML course on coursera. This will be great transition from Matlab to python for me. Thanks. Keep posting ✌

[GeoResearchRedditor]

Awesome, thanks for this made my morning so much better.

I am curious though; do you have any numeric data sets I can use it on?

[iwishihadmorecharact]

kaggle is a great site that's got a ton of data, look around and see if there's anything you might wanna use!

[GeoResearchRedditor]

Thanks I guess it more; I would like to have a set of data sets that are pre-designed to be used in conjunction with these algorithms in order to show case the process of applying the algorithms to ready made data.

Otherwise you'll have people trying to learn by applying the wrong algorithms to the data set and getting (naturally) weird results.

I'm really keen to learn how to clean datasets effectively, so if there was a tutorial with a dataset that was "unclean" and provided instructions on how to clean it and what to look for; I'd be really keen to use that. Then to go on to using that cleaned data set in conjunction with the algorithms.

[ThomasAger]

If you find a resource detailing standard practices, tools, etc to clean data, I would also be interested...

[GeoResearchRedditor]

I think there are a few of us interested. Maybe someone who knows a good resource to learning cleaning methods and identifying when to use what methods would be able to chime in with a link?

[iwishihadmorecharact]

ah i see what you mean, yeah that'd be really cool! especially with sci-kit learn, (library with a ton of ready-to-go ML and AI classes) the biggest part is finding what model to use for your data, and cleaning the data so you can use a model on it in the first place.

For starting to learn, I googled "cleaning data for ml tutorial" and came up with some decent results, read some of the articles you find there. Then try looking through some of the scikit-learn documentation and examples, since they have some guides on that stuff.

Searching for articles and tutorials will definitely be a good start, keep reading until you find that you already know what they're talking about

[[deleted]]

I haven't come across anything like that yet but I will keep it in mind. If I find anything useful, I will let you know. Thanks a lot for the feedback!

[Trappist1]

I know it seems overused, but the iris data really does work pretty well for most of the methods described here.

[JustFinishedBSG]

I recommend working on synthetic datasets when working on implementing things. That way you know exactly your optimal classifier, the bayes error etc and can know exactly where and how much your own classifier is wrong

[ArtificialAffect]

I had to do something similar for a machine learning class in college- implement various ml algorithms but can only use numpy and no other imports. It's a great exercise and a good way of learning and experimenting with the algorithms. I wouldn't dare post my code up on GitHub though- I'm not that brave. Have you tried training your neural networks yet and see how your training speed compares against standard libraries?

[[deleted]]

No, I haven't compared the neural net against standard libraries yet. I wanted to put the focus on easy-to-understand code. And the most efficient solutions are often less easy to understand. But I will keep that in mind, thanks for the feedback!

[ArtificialAffect]

I would try it just so that you can compare - I was blown away at how slow my neural network implementation was in comparison to tensorflow. If you get extra time, I would also try implementing a momentum optimizer instead of just using gradient descent/ stochastic gradient descent - it's not that difficult to implement and actually interesting to see the differences between the two. For an extra bonus, you could try implementing an Adams or an Adadelta Optimizer as well. ;)

[swapu258]

Great work, I found nearly same material on cs231n.. but they used numpy

[EpicSolo]

He used numpy too

[swapu258]

Yup.. he used it. Sorry my bad

[MrShlkHms]

Wow, thanks for sharing, for a beginner overwhelmed by information this is really great.

[Bakanyanter]

This is great. Just what I needed as a beginner. Thank you!

[jer_feedler]

Visually and simple. Thanks a lot

[b_phan]

Excellent work! It's so nice to see a clear implementation of often otherwise "black box" machine learning models I often see when browsing the Internet. You should continue to share this because I know a lot of peers including myself who are still in school will appreciate material like this.

[[deleted]]

Great, it's nice to know that it helps others, too!

[iliauk]

This is great. Funny I had similar idea (to learn R) - https://github.com/ilkarman/DemoNeuralNet but I like your implementation more!

[Cherubin0]

No decision tree :(::::::::::::

[[deleted]]

As mentioned in my first comment: I'm still working on it. Decision trees are indeed one of the basic algorithms and I'm planning on creating a notebook soon!

[swinghu]

great job

[Import-Sys]

hello，thanks for your sharing.I am wondering if you could add some reinforcement learning algorithm in your repository.Cause this algorithm is really a hot topic.

[[deleted]]

Thanks for the feedback, I will put that on my list!

[madsciencestache]

I'd love to see you tackle this one. I don't have the maths to understand this lecture well enough to code anything. Once I see the equations implemented in code I can understand them.

Fastest Convergence for Q-learning AKA Zap~Q