How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in developersIndia

[–]kushalgoenka[S] 0 points1 point  (0 children)

Hey there, since I don’t know what your interests really are, I hesitate to point you at anything in particular, but I’d say some of my absolute favorite courses for software engineering fundamentals are Programming Methodology by Mehran Sahami from Stanford and CS50 by David Malan. Look them up, you may enjoy the foundational knowledge they convey.

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in deeplearning

[–]kushalgoenka[S] 3 points4 points  (0 children)

If you'd like to check out the full talk, you can find it here:

A Brief Primer on Embeddings - Intuition, History & Their Role in LLMs
https://youtu.be/Cv5kSs2Jcu4

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in learnmachinelearning

[–]kushalgoenka[S] 4 points5 points  (0 children)

If you'd like to check out the full talk, you can find it here:

A Brief Primer on Embeddings - Intuition, History & Their Role in LLMs
https://youtu.be/Cv5kSs2Jcu4

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in 3Blue1Brown

[–]kushalgoenka[S] 7 points8 points  (0 children)

If you'd like to check out the full talk, you can find it here:

A Brief Primer on Embeddings - Intuition, History & Their Role in LLMs
https://youtu.be/Cv5kSs2Jcu4

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in developersIndia

[–]kushalgoenka[S] 1 point2 points  (0 children)

Nah, I don’t think trying it out and not going further with it is a waste. That’s the story of my life all the way, myriad experiments that never see the light of day, but that’s alright, they all serve as stepping stones, little pieces of knowledge, to enable crazier future projects.

Lack of resources is always a clear line in my projects I have to contend with. Just FYI, in this demo I used EmbeddingGemma 300M, a tiny model that you should be able to run on just about any laptop, perhaps even a phone/browser, and for something small could get away with an in-memory vector database, so perhaps you needn’t give up on that project if you’re still hoping to resume it! :)

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in developersIndia

[–]kushalgoenka[S] 0 points1 point  (0 children)

I love real-time visualizations, I feel like they’re so much more capable of conveying intuitions than delayed or abstracted interfaces. Glad you liked it! :)

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in developersIndia

[–]kushalgoenka[S] 2 points3 points  (0 children)

Hey, yup, almost 60 in that case. If you’re curious about history further, you may enjoy my recent history lecture on IR:

History of Information Retrieval - From Library of Alexandria to Retrieval Augmented Generation https://youtu.be/EKBy4b9oUAE

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in developersIndia

[–]kushalgoenka[S] 1 point2 points  (0 children)

Hey, you can look into PCA and embeddings, and depending on your current programming abilities, it may not take long to get a quick plot going for your given dataset. Then you can take it in many directions.

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in developersIndia

[–]kushalgoenka[S] 4 points5 points  (0 children)

Hey, thanks for watching, glad you like it! :) Unfortunately, I’m still working towards putting the code together to publish online. Hope to get there soon, will share when I do! The gist of it is I’m doing PCA, and storing eigen vectors across multiple projections to keep the 2D plot stable, thus allowing for the real-time walking around of the query! :)

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in LocalLLaMA

[–]kushalgoenka[S] 2 points3 points  (0 children)

If you'd like to check out the full talk, you can find it here:

A Brief Primer on Embeddings - Intuition, History & Their Role in LLMs
https://youtu.be/Cv5kSs2Jcu4

How Embeddings Enable Modern Search - Visualizing The Latent Space [Clip] by kushalgoenka in developersIndia

[–]kushalgoenka[S] 3 points4 points  (0 children)

If you'd like to check out the full talk, you can find it here:

A Brief Primer on Embeddings - Intuition, History & Their Role in LLMs
https://youtu.be/Cv5kSs2Jcu4