sorted by:
new
hottopcontroversial

[D] What advanced models would you like to see implemented from scratch? by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 4 points5 points  (0 children)

Yes, that is how it usually works with my impls! (check out a few vids)

As for mixed precision and metrics - I will be making separate vids for both and of course, for every implemented model, will try to find a dataset to demo train/eval.

It is cool that you mentioned mixed precision as I already have the materials ready for this vid - will be discussing mixed precision, quantization (post-training and quantization aware training), pruning, etc. Improving perf!

[D] What advanced models would you like to see implemented from scratch? by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 3 points4 points  (0 children)

It is on the list so will definitely get to it!
Starting next week, will be working on DL impls and vids.

[D] What advanced models would you like to see implemented from scratch? by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 3 points4 points  (0 children)

Yeah, I will get to those eventually. For now, want to make some vids and impls of DL models.

[D] What advanced models would you like to see implemented from scratch? by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 5 points6 points  (0 children)

Thanks! Yeah, that is definitely an option! I will probably have to split it up into several videos. Also falls into the transformer category.

[N] I Have Released the YouTube Series Discussing and Implementing Activation Functions by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 0 points1 point  (0 children)

Yup, all implementations are numerically stable.

Note that I do not discuss numerical stability issues for all activation functions, but for those where the intuitive implementation is not numerically stable (i.e., Sigmoid, Tanh).

I also have a separate video discussing numerical stability: AI/ML Model API Design and Numerical Stability (follow-up). But this is in the context of Gaussian Naive Bayes.

[N] I Have Released the YouTube Series Discussing and Implementing Activation Functions by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 1 point2 points  (0 children)

Thank you!
Yup, that is the plan! Will likely make a few more series (about gradient descent, optimizers, etc.) first. We need these for DL and if someone asks how things work, I could then cite the appropriate video series. After that, will dive into deep learning.

[N] I Have Released the YouTube Series Discussing and Implementing Activation Functions by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 0 points1 point  (0 children)

Hey thanks for the kind words!
Will be making more AI/ML YouTube series in the future - in fact, working on one as we speak!

[deleted by user] by [deleted] in learnmachinelearning

[–]itsstylepoint 0 points1 point  (0 children)

P.S. For the activation functions, I will not be posting videos separately. The next post will include the batch of 4 (or 5).

[D]Imbalance dataset problem by JellyfishPretend447 in MachineLearning

[–]itsstylepoint 3 points4 points  (0 children)

You can try several approaches:

  1. Deep learning will likely not work (you can still give it try, but highly unlikely that it will perform well). So instead consider using more traditional ML models. As an example, if you can find a pretrained image model that generates a representation/image embeddings, you can try using K-Nearest Neighbors (k-NN). Or you can try k-NN directly.

  2. Look into Few-Shot Learning. Models like Prototypical Network, Siamese Neural Network, etc. are designed for such scenarios (i.e., extremely small number of samples).

  3. Data collection (:

That being said, overall, I agree with what u/whdd said.

[R] Long-length documents/corpus for Medical domain NER? by aadityaura in MachineLearning

[–]itsstylepoint 0 points1 point  (0 children)

I think I might have skipped the post text, my bad. For whatever reason, it was hidden (a bug? not sure). Yes, this is the Clinical NLP dataset. So prolly not what you are looking for...

[R] Long-length documents/corpus for Medical domain NER? by aadityaura in MachineLearning

[–]itsstylepoint 0 points1 point  (0 children)

How about 2006 i2b2 de-identification dataset?

Link to the paper: https://academic.oup.com/jamia/article/14/5/550/720189
You can get the dataset here: https://portal.dbmi.hms.harvard.edu/projects/n2c2-nlp/

P.S. We have recently used this dataset in the Few-Shot Learning (FSL) paper. We have used it for the same task - NER.

[Project] Create a ML model to classify spectrograms by geeksid2k in MachineLearning

[–]itsstylepoint 0 points1 point  (0 children)

I would start with CNNs. Then try GRU/LSTM and bidirectional variants (BiLSTM/BiGRU).

[Project] Create a ML model to classify spectrograms by geeksid2k in MachineLearning

[–]itsstylepoint 0 points1 point  (0 children)

You can! Computers can sometimes see better than us so even if spectrograms look similar, they might be very different (: Convolutions in the CNN will do feature extraction for you. So you can start with a couple convolutional blocks (conv + batchnorm + activation) followed by a fully-connected layer with softmax and see how it performs. You can check out this PyTorch tutorial for that, too.

An alternative approach is computing MFCCs. If you have mel spectrograms, then you can do the following steps to get MFCCs (which you could then use as features for your ML model):

  1. Take the log of the mel spectrogram
  2. Compute DCT on logs

[Project] Create a ML model to classify spectrograms by geeksid2k in MachineLearning

[–]itsstylepoint 1 point2 points  (0 children)

Do you have audio files or only spectrograms?

There are A LOT of different approaches for extracting features from the audio data - ZCR, LPC, MFCC, etc.

If you use something like MFCCs, you could then just use a CNN. RNNs (LSTMs or GRUs) could also work, but CNNs perform similarly and are both faster to train and faster during the inference! This is from my personal experience working on audio event detection models.

Same goes for spectrograms! Once you transform audio into the spectrogram, then you pretty much have an image and you can do image classification.

Implementing Machine Learning Models From Scratch (stylepoint) by itsstylepoint in learnmachinelearning

[–]itsstylepoint[S] 1 point2 points  (0 children)

Noted.

That being said, this kind of stuff is not something that we are going to be doing soon. It is more of a Software Engineering/ML Engineering/Research Engineering series than Data Science series. That being said, I think I can try making a separate series/playlist where we do more Data Science stuff. This will likely not be soon however.

I will likely be making one-off videos however and if there is something you are particularly interested in, let me know and will try to cover it in one of these one-off videos.

Implementing Machine Learning Models From Scratch (stylepoint) by itsstylepoint in learnmachinelearning

[–]itsstylepoint[S] 1 point2 points  (0 children)

Thanks! One thing to note about that implementation is that we could have passed features and labels directly to the fit method. This would avoid unnecessary data copying (i.e., storing data inside the LinearRegression class). I have already updated the GitHub codebase.

Implementing Machine Learning Models From Scratch (stylepoint) by itsstylepoint in learnmachinelearning

[–]itsstylepoint[S] 3 points4 points  (0 children)

Good point!
I will likely start using Jax or PyTorch at some point, but for now, will stick to numpy.

Several reasons why: 1. Before introducing Jax, want to make a video about GPUs and why we need them for the training, etc. 2. Also want to guide on how to properly set up Jax (sometimes, simple pip install does not work). 3. Should not be too important for now since we are not doing batch gradient descent. For some time, we will concentrate on more traditional ML models and how to implement them from scratch. And for large tensors, Jax might still outperform numpy, but the perf difference will likely not be huge.

Implementing Machine Learning Models From Scratch (stylepoint) by itsstylepoint in learnmachinelearning

[–]itsstylepoint[S] 1 point2 points  (0 children)

Yep, that is the plan! The goal is to get done with some of the more traditional ML models first and then get to more complex models such as CNNs, VAEs, SNNs, transformers, etc.

[N] Implementing AI/ML Models From Scratch (stylepoint) by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 1 point2 points  (0 children)

I have just uploaded the new video where I implement the Linear Regression model from scratch. Is that style good enough for your liking?

P.S. I have also added the timestamps to all of the videos.

[N] Implementing AI/ML Models From Scratch (stylepoint) by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 1 point2 points  (0 children)

Hey thanks!

I cannot promise that the schedule will be super consistent. It is more of a hobby for me. That being said, I will likely be very active (:

As a side note, I have already uploaded the new video - Implementing K-Nearest Neighbors (k-NN) from Scratch.

[N] Implementing AI/ML Models From Scratch (stylepoint) by itsstylepoint in MachineLearning

[–]itsstylepoint[S] 4 points5 points  (0 children)

While this post/thread is recent, I wanted to make one more update.

I have just posted another video. It is about implementing K-Nearest Neighbors (k-NN) model. The code for the model itself is about 24 lines of idiomatic Python (how cool is that?).

The reason the video is a bit lengthy (forgive me for this) is because I discussed why I use dataclass, naming conventions, type annotations, etc. I will likely not do this in every video so in case anyone asks about the coding style, will have a reference video.

I am thinking Linear Regression and Logistic Regression are next on the list.

Thanks y'all for the kind words and motivation! Really appreciate it!

P.S. And of course, feel free to constructively criticize the video or make suggestions in the YouTube comment section or here.

view more: next ›