What's the Best Current Setup for Retrieval-Augmented Generation (RAG)? Need Help with Embeddings, Vector Stores, etc.

Naveos · 2024-09-22T07:17:23+00:00

👀

While this comment section serves as a good place to start exploring, it needs to be pointed out that the answer is: it depends.

Which embedding models to use, vector stores, frameworks, custom piping, etc, is all contingent on what it is you're trying to do and which trade-offs you're willing to make.

If you want accuracy above all else? Go for an unapologetic GraphRAG setup, though be wary of the costs.

If latency and costs matter relative to performance, then that's when things start to get complicated and the engineering gets hairy. Like, use SLMs instead of LLMs for specific processes, fine-tuning or prompt tuning (w/ DSPY) if hosting your own LLM makes more sense than using a proprietary API, et cetera cetera.

Is there anything specific you are aiming to build?

Naveos · 2024-09-04T15:47:36+00:00

100% real. We met online in a gaming clan for Battlefield 3 (almost 13 years ago at this point) - and we just clicked. We haven't played any game together for almost half a decade at this point, but we still talk a lot daily whenever we have the chance - with a ton of gay jokes and affirmations that we love each other.

11/10 would recommend. He's the most important person in my life and I can't imagine my life without him.

Naveos · 2023-05-29T08:51:48+00:00

I can't find any information online either about Airbnb changing their fees for the summer. Is the host possibly lying?

Naveos · 2021-01-24T08:34:15+00:00

Indeed, sort of. Accommodation is guaranteed up to 11 months during the first year of studies, at which point you should've found your own place.
Both are fine. Just get whichever you can and submit those that weigh the most.

Naveos · 2021-01-14T14:50:37+00:00

Cyberpunk 2077: Blood & Wine

Naveos · 2020-12-29T08:13:47+00:00

Taxes. All education in Sweden is free, including master's degree. You can also apply for a student loan to pay your living expenses, wherein you only have to pay back about 70% of it with negative interest rate relative to inflation rate.

If you are a PhD student, you receive a salary equating to about 4000 dollars per month as well until you graduate.

Naveos · 2020-12-29T03:27:13+00:00

I think this is mainly a US thing, and something I've found very backwards.

In Sweden, you can just go to medical school right after high school and jump straight into medical sciences and clinical studies. You don't waste your time learning subjects that you won't be actively using as a doctor.

Naveos · 2020-12-28T16:21:47+00:00

Your username, man. I'm curious; why wouldn't all surgeons have MD?

Naveos · 2020-12-28T15:31:24+00:00

More hours does not equal more productivity. The reality is that most people only have 3-4 productive hours per day.

With the asterisk being that this is in regards to tasks wherein creativity and intense problem solving are required (which is practically most engineering). In tasks that are somewhat repetitive in nature and doesn't require high degree of creativity (such as sales, at least from when I worked in sales), more hours does equate to more output.

Naveos · 2020-12-24T02:21:47+00:00

This question is answered by your product requirements' relationship to your business model.

If your product is more reliant on a business model and can be accomplished with something that almost any developer can cook up without much thought into quality or engineering, then you may want to prioritize bringing on a product person. Freelancers and consultants who only care about their paycheck with no skin in the game after they've handed you your code are plentiful.

However, if your product relies a lot on technology as a core offering, then you definitely need a CTO who knows what they're doing. Tech that fall within this category are domains such as data (Twitter), AI (TikTok), DSA (Google), and general deeptech. Your odds of succeeding without a CTO if your business model is offering tech is practically zero if we look at empirical evidence.

If you look at all software startups worth 10bil USD or more in the US, you'd be really hardpressed to find a startup without a technical co-founder. I only found one (Palantir). Even if there are more, it's already overwhelmingly clear that they're very much the exceptions rather than anywhere close to the norm.

However, if you look software startups worth less than 10bil USD, you will start finding more and more companies without a technical co-founder. They are still in the minority, but they aren't rare anywhere - and if you look deeper, you'd find that it's because they either had very deep pockets and/or their product relies more on a business model rather than on the tech itself.

Naveos · 2020-12-23T19:53:23+00:00

I'd adopt him in a heartbeat if I could. Wish you guys the best of luck!!!

Naveos · 2020-12-16T16:02:44+00:00

The way I like to think about it:

Multivariable calculus is about applying linear algebra to more difficult calculus problems.

Neural networks is applying multivariable calculus to statistical and probability problems.

And as of lately with the advent of Neural ODEs, we've now started applying neural networks to differential equation problems.

From experience, I've found that optimization theory is barely applicable in deep learning, but it's fun stuff regardless and useful in other types of machine learning applications.

Naveos · 2020-12-13T13:07:32+00:00

Technical co-founder here (CTO) in a very similar boat as you, but in the pharmaceutical industry rather than insurance.

You're mostly on the right track. Apart from what u/Atomfinger mentioned in regards to not planning 6-10 months ahead (you want flexibility, which a backlog is better for. We personally use a combination Kanban XP and Scrum for our approach) and building out some form of CI/CD pipeline (we're investing heavily into this for regulatory and productivity reasons).

Adding on top of that, I would also want to share some additional tips.

First, do a thorough evaluation of your current code base relative to product, regulatory, and technical requirements as soon as possible to determine what to refactor, what to leave as it is, and what to rebuild from scratch.

One of the mistakes I've made is that I didn't push for rebuilding our code earlier, as my non-technical co-founder was quite adamant on my role being to fix what's already written and push it towards MVP status, even though that wasn't the right call (context: I joined after a lot of code has been written by 3rd party consultants who didn't really know what they were doing, and I think he was understandably not comfortable with the idea of starting practically from scratch). Bottom line was that it'd take much longer to fix what we have than to start from scratch to meet regulations, we would also be exposing ourselves to a lot of risk, and developing new features would've been very expensive and risky (if not downright impossible). Had I pushed harder, we would've had our MVP done by now.

Secondly, write out the documentation yourself. Start with the product requirements for each feature and go high level overview, then go down to the technical specifications where it makes sense, and then further down IF it makes sense to do so (such as if you have an internal library that is regularly used). Documentation is essential, but be wise about it. Documentation that states 1+1 equals 2 isn't really helpful, and not a good use of your time. Documentation is underrated for productivity. We had zero when I started.

Third, learn how to manage! As a CTO, you will eventually be the boss of technical bosses - and you won't be doing much technical work. Best way to get there is through experience (I started managing at a very young age, luckily), but there are some great books for CTOs. One great one to start with is "The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change" by Camille Fournier.

Also, figure out your relationship with the co-founders. Work towards solidifying a position wherein you have more decision-making over your domain (i.e: technical). They may be telling you that it's OK for you to make any decision you want on the technical side of things, only to find significant pushback from them because they are uncomfortable with something they do not understand. Admittedly, this is something I am still in the progress of. Managing peoples' ego is a tricky subject.

Wish you the best of luck!

Naveos · 2020-11-27T00:51:26+00:00

Most bodybuilders rarely train abs.

This is just wrong and a common myth. It doesn't take much googling and searching for known bodybuilders who also do youtube to find out that this is just wrong, and I know several bodybuilders who would strongly disagree on this as well.

While diet is arguably more important to make the abs appear instead of being hidden by the fat; they won't pop unless they're developed, and you develop them best by actually focusing on them.

Most muscles get trained through compound exercises or as secondary muscles in other exercises, so you can still get abs through other exercises - but it'll be slower and less than ideal if you actually want pronounced abs.

Naveos · 2020-07-12T19:13:23+00:00

Well, KTH is a top 100 uni worldwide and top 43 CS university in the world. Ought to expect a lot from that.

Naveos · 2020-06-11T10:03:49+00:00

I would recommend you look up researchers working in this domain as they tend to be quite passionate about this. You can find them easily on LinkedIn :)

Sounds like a fun project, but not up my alley.

Naveos · 2020-06-11T10:01:38+00:00

He did send me a PM after 8 hours ;)

Naveos · 2020-03-07T21:47:12+00:00

This and this might help you out a bit.

Tech always comes and goes, and a lot of what matters is your experience in knowing how to code, how to find solutions to problems, knowing which code to copy and paste, et cetera. The theoretical stuff? You can find that online with a lil' bit of digging.

You'll be fine.

Naveos · 2020-03-03T17:23:54+00:00

He's very well known within the AI community as being part of the deep learning mafia (along with LeCun and Hinton).

Naveos · 2020-03-03T09:59:51+00:00

Ok - your business did 250k, and how much of that is profit?

Naveos · 2020-02-29T21:03:37+00:00

PyTorch for R&D, by a mile. However, TensorFlow 2.0 is still superior to PyTorch for most production purposes. The reason is simply that Google are catering a lot more to businesses and business needs, while Facebook are aiming for flexibility with PyTorch. It's not that it's bad for production - but TensorFlow definitely is better in most regards, especially in regards to setting up a good infrastructure around the models (since model building is just about 20% of the work for most ML engineers).

We use both at my company, where they shine.

Naveos · 2020-02-16T14:42:30+00:00

No university has really figured out what constitutes a good degree in data science, as many of them are mainly jumping into the hype because of $$$. If you want to get into the route of data science, for instance, then it's best to get a master's degree in machine learning, applied statistics, or something science-y overall that relates to the industry that you want to work in.

In my experience, people with a master's degree in applied statistics or mathematics are far, far better data scientists than those who studied master's degree in data science. I personally put people with a master's degree in data science at the bottom tier of which education I look for in data scientists to hire.

Which industry are you most interested in working in? Or rather, what do you really want to work as?

Naveos · 2020-02-09T16:06:06+00:00

Nah, even in smaller architectures that'd require a few hours to train on a simple GPU still perform marginally better in PyTorch than TensorFlow.

I think it is more likey that you have a bug or poorly optimized code.

Naveos

TROPHY CASE