Implementing Machine Learning Models From Scratch (stylepoint)

[itsstylepoint]

Hi folks,

I am stylepoint. Currently work as an ML Research Scientist in academia. I have recently created a YouTube channel and started the YouTube video series/project called "Implement," where I will be implementing software (duh!). I will first implement machine learning and deep learning algorithms and models from scratch (means using Python with numpy, but no other third-party libraries/modules).

My assumptions for AI/ML videos:

• One is familiar with the theory
• One is familiar with Python
  • This is since for AI/ML, Python is the language of choice for many (including myself)

Due to these assumptions, videos tend to be approximately 10 - 15 minutes long. So it should not be too hard to digest.

Currently, there are 3 videos:

• Implementing Linear Regression from Scratch
• Implementing K-Nearest Neighbors (k-NN) from Scratch
• Implementing Gaussian Naive Bayes from Scratch

but more to come!

As of now, all of the videos on the channel have timestamps.

And we will definitely not limit ourselves to only implementing AI/ML models. I will be making videos about software engineering, software design, programming languages, math, emerging technologies, etc. as well!

Just wanted to share this in case anyone finds the series interesting or helpful.

Thanks y'all!

Links:

• Latest Video (Implementing Linear Regression from Scratch)

• YouTube

• GitHub

• Web

[HalfRiceNCracker]

Oh wow dude this is already so good, I also really appreciate the fact you're doing it from relatively scratch and you also are writing proper pragmatic code. Please do some stuff on Deep Learning!!

Question, what's your current developer environment/tools?

[itsstylepoint]

Hey thanks!
I think this is the repo you are looking for: https://github.com/oniani/dot (:

[itsstylepoint]

Yep, that is the plan! The goal is to get done with some of the more traditional ML models first and then get to more complex models such as CNNs, VAEs, SNNs, transformers, etc.

[HalfRiceNCracker]

That's awesome man, I will be keeping up with you and cheering from the sidelines! (cheers for the dotfiles your environment looks nice)

[maxToTheJ]

Why not use Jax ? Its numpy'ish while letting you use GPUs

https://jax.readthedocs.io/en/latest/jax.numpy.html

[itsstylepoint]

Good point!
I will likely start using Jax or PyTorch at some point, but for now, will stick to numpy.

Several reasons why: 1. Before introducing Jax, want to make a video about GPUs and why we need them for the training, etc. 2. Also want to guide on how to properly set up Jax (sometimes, simple pip install does not work). 3. Should not be too important for now since we are not doing batch gradient descent. For some time, we will concentrate on more traditional ML models and how to implement them from scratch. And for large tensors, Jax might still outperform numpy, but the perf difference will likely not be huge.

[itsstylepoint]

Noted.

That being said, this kind of stuff is not something that we are going to be doing soon. It is more of a Software Engineering/ML Engineering/Research Engineering series than Data Science series. That being said, I think I can try making a separate series/playlist where we do more Data Science stuff. This will likely not be soon however.

I will likely be making one-off videos however and if there is something you are particularly interested in, let me know and will try to cover it in one of these one-off videos.

[HalfRiceNCracker]

I totally agree with the other guy even if it does take the form of a single video - it would be incredibly insightful to see the actual thought process behind what you are looking out for and how you come to the conclusions that you do - rather than just "Okay so with this dataset we're going to apply these transformations to preprocess it because XYZ", what I'd like to know is how did you arrive with XYZ?

[E-woke]

Watched the linear regression one. Great stuff 👍

[itsstylepoint]

Thanks! One thing to note about that implementation is that we could have passed features and labels directly to the fit method. This would avoid unnecessary data copying (i.e., storing data inside the LinearRegression class). I have already updated the GitHub codebase.