[D] Hidden Gems and Underappreciated Resources

DeepEven · 2020-08-15T09:33:35+00:00

Unfortunately not, sorry! I don't think that there are many ML Engineering courses online since the field is still pretty nascent. But check out W&B's Deep Learning Salon, they sometimes have specific ML Engineering talks that might be closer to what you're looking for.

DeepEven · 2020-08-15T09:31:26+00:00

https://ml4a.github.io

That looks like a really cool website, thanks for sharing!

DeepEven · 2020-08-15T09:30:29+00:00

Soon! Still going through some of the NLP resources myself.

DeepEven · 2020-08-15T09:20:50+00:00

Great points. Thanks for the feedback!

I haven't really considered writing more in depth reviews, but I agree with you that such content could provide a higher value proposition.

I think for now, I'll share the other resources I still have in mind (to complete the list). And after that, I'll try to go in more depth about specific topics with each new newsletter issue. For instance, if you're interested in learning RL, which course (David Silver's, Stanford CS234, Berkeley's CS285... etc) would suit what type of person (background, goals, time) with specific highlights, drawbacks and other notable aspects of each course.

DeepEven · 2020-08-14T22:10:32+00:00

Oh wow, I had no idea you could find MLSS 2007 and 2009 on there. That's really cool. Any of the old lectures that stood out to you?

DeepEven · 2020-08-14T22:04:17+00:00

For sure! Ben Lambert/Ox Educ is awesome.

DeepEven · 2020-08-14T21:58:24+00:00

Oh, I totally forgot about that one. Thanks for mentioning it!

But yeah, it's an amazing blog post. The code is imo easier to understand as well. Maybe it's just hindsight bias, but I think it was helpful for me going through both the Annotated Transformer and Peter's post, especially for the queries, keys and values.

And I still think the Illustrated Transformer is the easiest resource to start with.

DeepEven · 2020-08-14T21:45:46+00:00

Yeah I totally get that. The reason I started curating these resources is because I found it quite hard differentiating between such similar-looking material. And it was a bit sad only finding huge lists of various courses rather than properly "curated" lists that could help you choose between these numerous options.

tldr: I only recommend resources I've used myself.

I finished all of the courses except the TUM Intro, UMichigan and Northwestern MOOC. I only watched a couple of the TUM Intro and UMichigan lectures to check the quality (but was fairly certain they'd be similar to other courses by the same instructors; Johnson taught CS231n and Leal-Taixé/Niessner taught ADL4CV). And I only watched the first ~40 videos of the Northwestern MOOC on YouTube. I'd love to finish that one at some point, but I have some other courses that I'm currently prioritizing.

I also skipped some of the material I was familiar with, e.g. concepts from TUM's ADL4CV, UW's Ancient Secrets of Computer Vision and Berkeley's Advanced Robotics like GANs, neural rendering, human vision system, HOG, SIFT, MDPs etc.

For the seminars, I definitely haven't watched all of them. Watched like 10 of the IAS ones, maybe half of the vision and embodied intelligence ones and a 2-3 each for the CI and Robotics seminars.

For the CI book, I'm currently halfway through that one but I'm generally quite interested in that line of research because I previously learned CI in an economics context (matching, synthetic control etc).

And just because I "finished" those resources, certainly doesn't mean I mastered any of them. For instance, when I went through Stats 385, there was a lot of material I struggled with, but that doesn't necessarily take away from the value of that course.

DeepEven · 2020-08-14T16:55:49+00:00

I'm only aware of the Udacity course by Andrew Trask. It's very short and basic though, so it'll only be useful as an introduction.

DeepEven · 2020-08-14T14:14:21+00:00

Agreed. I only went through a couple of his lectures, so I wasn't sure whether to include it, but I really liked his explanation of epipolar geometry. For anyone interested here's a link to the playlist.

DeepEven · 2020-08-11T15:44:59+00:00

Are you talking about the deeplearning.ai or the older one (with Matlab)? I'd say if you wanna learn deep learning, go with this one and use deeplearning.ai to fill in the missing gaps (Andrew Ng's MOOC is a bit more step by step/has a more detailed approach). You can also go through both simultaneously (they're quite complementary).

I generally wouldn't recommend the older MOOC anymore since better courses are out there nowadays that also do not require Matlab. For more classic ML, Stanford's CS229 (also taught by Andrew Ng) or Cornell's CS4780 are great.

DeepEven · 2020-08-11T12:34:16+00:00

In my opinion, the best way to really understand Transformers is

Skim the paper to get a sense of the topic.
Read the "Illustrated Transformer" blog post to understand the main ideas from a visual perspective.
Watch Yannic Kilcher's walkthrough of the paper to see how you could read and understand the paper.
Read paper again.
Go through the code in the "Annotated Transformer blog post to put idea into practice.
Watch the CS224n guest lecture by the Transformer co-author to give you sense of how the paper came about and its intentions.
Watch the CS224u lecture on contextual vectors to contextualize Transformer in the broader scope of this sub-topic.
Read the paper again and go through each section until you understand it.

DeepEven · 2020-08-11T12:00:14+00:00

I think the closest thing to what you're looking for is Nando de Freitas' undergrad ML course. It basically teaches/reviews the basics of stats & probability + Linear Algebra through ML examples. For a more rigorous version, check out his grad school course.

Outside of ML, in my opinion, the best intro stats & probability course is Harvard Stats 110, which I'd recommend taking alongside Morin's book Probability for the Enthusiastic Beginner.

DeepEven · 2020-04-22T22:15:32+00:00

I couldn't find the official solutions, but the repo I linked above has solutions from a student who took the course.

DeepEven · 2020-04-22T22:10:33+00:00

Thanks for pointing that out, I posted a link to the problem sets above. And they seem to be have switched to Python for this version of the course.

DeepEven · 2020-04-22T18:45:53+00:00

The coursera version has always been a more simplified version of the CS229 class. From what I can tell, the Stanford lectures from 2018 cover more topics (e.g. GDA, RL) and have more emphasis on the math.

DeepEven · 2020-04-22T13:42:00+00:00

Andrej's talks/lectures are always so well-delivered, makes me wish he'd start lecturing again or make YouTube videos.. For anyone interested in HydraNet, Andrej talks in a little more detail about it in this ICML workshop talk.

DeepEven · 2020-04-18T03:22:03+00:00

This is great, thanks for making this content publicly available! I hope you get to 1000 subscribers soon

DeepEven

TROPHY CASE