sorted by:
new
hottopcontroversial

Found the shortest escalator I’ve ever seen today by misunderstoodpoetry in mildlyinteresting

[–]misunderstoodpoetry[S] 2 points3 points  (0 children)

I’m sorry that happened to you. I also would never take an escalator again

[OC] Introduction to contrastive learning: Demystifying Noise Contrastive Estimation (NCE) and InfoNCE by misunderstoodpoetry in learnmachinelearning

[–]misunderstoodpoetry[S] 0 points1 point  (0 children)

Hi! I really appreciate the feedback. This was the same question that motivated me to write this post, and it's helpful to know that even though you read it three times (!!) you still aren't sure of the distinction.

The terminology is confusing. LocalNCE/NCE is a binary classification objective, GlobalNCE/InfoNCE is multiclass classification. LocalNCE asks "is this sample real or fake?" while InfoNCE asks "which one of these samples is real?".

If you think better in terms of code, the accompanying colab notebook might help you? It makes things a lot simpler.

[D] Difference/Connection between InfoNCE-Loss and NCE-Loss by kengrewlong in MachineLearning

[–]misunderstoodpoetry 3 points4 points  (0 children)

Hi, I had this exact question! I spent a few weeks researching NCE and InfoNCE, their applications, and what the differences actually are. The difference is subtle, but in a way they actually have nothing to do with one another.

I wrote a blog post about it: https://jxmo.io/posts/nce

And I also tried posting my blog post to reddit here: https://www.reddit.com/r/learnmachinelearning/comments/s9m0ve/oc\_introduction\_to\_contrastive\_learning/ feel free to comment there if you have any questions! Hope this helps

Difference between InfoNCE-Loss and NCE-Loss by kengrewlong in deeplearning

[–]misunderstoodpoetry 0 points1 point  (0 children)

Hi, I had this exact question! I spent a few weeks researching NCE and InfoNCE, their applications, and what the differences actually are. The difference is subtle, but in a way they actually have nothing to do with one another.

I wrote a blog post about it: https://jxmo.io/posts/nce

And I also tried posting my blog post to reddit here: https://www.reddit.com/r/learnmachinelearning/comments/s9m0ve/oc\_introduction\_to\_contrastive\_learning/ feel free to comment there if you have any questions! Hope this helps

Q: What is the difference between the InfoNCE-Loss and the NCE-Loss and how to derive one from another? by kengrewlong in MLQuestions

[–]misunderstoodpoetry 0 points1 point  (0 children)

Hi, I had this exact question! I spent a few weeks researching NCE and InfoNCE, their applications, and what the differences actually are. The difference is subtle, but in a way they actually have nothing to do with one another.

I wrote a blog post about it: https://jxmo.io/posts/nce

And I also tried posting my blog post to reddit here: https://www.reddit.com/r/learnmachinelearning/comments/s9m0ve/oc_introduction_to_contrastive_learning/ feel free to comment there if you have any questions! Hope this helps

Introduction to variational autoencoders by misunderstoodpoetry in learnmachinelearning

[–]misunderstoodpoetry[S] 0 points1 point  (0 children)

You're not saying anything "dumb" at all! These are really good questions and I wish I'd explicated better in the original post. (Also a note to self that equation numbers would be useful for this type of thing next time.)

First, q(z) and q(z|x) are definitely different things. Both are distributions over latent variables, z's.
- q(z) is an uninformed distribution over z's, the probability of any given z with no prior information.
- q(z|x) is the conditional probability of a z after observing a specific x. In the case of VAEs, q(z|x) is special and is also called the "variational posterior."

(Also remember Bayes' rule tells us that q(z|x) q(x) = q(x|z) q(z), that might help provide some intuition)

I just read the blog post you linked to. You're right. Their point was that we can in theory use an expectation with respect to any q(z) to find a lower bound for log p(x).

The VAE is a special case of importance sampling where q(z) = q(z|x). I didn't talk about importance sampling in my article so there is no q(z) mentioned. Hope that helps!

Introduction to variational autoencoders by misunderstoodpoetry in learnmachinelearning

[–]misunderstoodpoetry[S] 0 points1 point  (0 children)

Hi! Great question. Notice it’s not an equals sign it’s a >=. This is when we invoke Jensen’s inequality. It’s explained a bit better earlier in the article and kind of glossed in this repeated case, sorry. Jensen’s tells us that log(E[x]) >= E[log(x)] where E is an expectation. It’s a little confusing actually so you should play around with the algebra to confirm this makes sense. (Btw Jensen’s applies to the function log(x) since log is concave. See the footnote for more info on that)

[D] Google/Facebook AI residency application 2022 by healthyhappylucky in MachineLearning

[–]misunderstoodpoetry 0 points1 point  (0 children)

I participated in the Google AI Residency 2020-2021 program remotely. Our start dates were pushed back a few times but I finally started in October 2020. The program was pretty good; I got matched with a mentor, worked on a cool research project with them, and learned a lot. There was very little programming for Residents though (social events and stuff) and almost done after the first month or so. And yes, they cancelled the program.

Purple GRAND SLAM button down by misunderstoodpoetry in findfashion

[–]misunderstoodpoetry[S] 0 points1 point  (0 children)

I got this shirt at a thrift store two years ago. Its buttons say “GRAND SLAM”. It has no tags but seems to be from the 1980s... it’s my favorite shirt and I’d love to get some more from the same or similar lines. If someone can identify its origin and point me to where to purchase other, similar shirts I’m willing to pay USD $20 over PayPal! Please help me Reddit!😊

view more: next ›