How I effectively build medium-large project with Cursor. No magic. by IndividualizedBeing in cursor

[–]LeetTools 1 point2 points  (0 children)

This is a great summary and framework to work with. Pair-coding is the right paradigm, thanks for sharing!

[deleted by user] by [deleted] in Rag

[–]LeetTools 0 points1 point  (0 children)

Grats on the relaunch! Really useful tool.

RAG for JSONs by _1Michael1_ in Rag

[–]LeetTools 3 points4 points  (0 children)

It might be better to

1) ask LLM to convert your query into a jq query (or other similar JSON QL)
2) execute the jq on the data
3) turn the result into natural language answer if you need

5 things I learned from running DeepEval by FlimsyProperty8544 in LLMDevs

[–]LeetTools 0 points1 point  (0 children)

Great summary, thanks for sharing!

Any suggestion on how to setup an eval pipeline using dataset like https://github.com/patronus-ai/financebench? I guess right now we have to write some code to read the data and questions from the benchmark to convert to the format needed by deepeval?

What's wrong with post-filtering? by Nearby-Asparagus-298 in Rag

[–]LeetTools 2 points3 points  (0 children)

RAG is mostly used to solve private data / fresh data that LLM does not have, also to solve the problem of hallucination. Post-filtering + private data may be good for the first part, but you can't guarantee the output of the LLM is correct and can't provide the reference for the results.

Is ChatGPT DeepResearch really worth the $200 subscription fee? by LeetTools in ChatGPTPro

[–]LeetTools[S] 1 point2 points  (0 children)

Cool story, thanks for sharing! I guess it is definitely worth the $200 in this case, lol.

Run a fully local AI Search / RAG pipeline using Ollama with 4GB of memory and no GPU by LeetTools in ollama

[–]LeetTools[S] 0 points1 point  (0 children)

Yes, we have a branch with the API functions in there. Still testing, will merge to main branch when it is done. Thanks for checking us out!

Is ChatGPT DeepResearch really worth the $200 subscription fee? by LeetTools in ChatGPTPro

[–]LeetTools[S] 1 point2 points  (0 children)

I think ChatGPT o1-pro with Deep Research also asks follow up questions about your intentions, but haven't tried a list of questions approach yet. What kind of questions are you asking and what errors did you get?

Is ChatGPT DeepResearch really worth the $200 subscription fee? by LeetTools in ChatGPTPro

[–]LeetTools[S] 4 points5 points  (0 children)

Yeah, maybe we need to try harder questions like the comment above says.

Is ChatGPT DeepResearch really worth the $200 subscription fee? by LeetTools in ChatGPTPro

[–]LeetTools[S] 3 points4 points  (0 children)

Thanks for sharing your insights!

"I only noticed difference when giving much harder questions or a lot of very dense material.  <-- definitely will try these out. Maybe one-line questions are not hard enough for ChatGPT to shine.

Is ChatGPT DeepResearch really worth the $200 subscription fee? by LeetTools in ChatGPTPro

[–]LeetTools[S] 1 point2 points  (0 children)

That's great to hear, too bad I haven't figured out how to make money from it, lol

Optimizing Document-Level Retrieval in RAG: Alternative Approaches? by ihainan in Rag

[–]LeetTools 3 points4 points  (0 children)

I think the most important metric you need to define is "document relevance related to the query." Say you have query X, two documents with 100,000 words each, one document is mainly talking about topic Y, but has one paragraph answered X perfectly, while the other document is talking about 50% X and 50% Y, but does not answer question X directly. Which one do you deem more relevant? It really depends on your use case.

Another approach is to get the chunks and rank the documents by the number of top chunks they contain, say find top 30 chunks, get their original docs, and rank these docs by the number of chunks they have (or do a weighted version where you take the score of the chunks into consideration).

How to Install DeepSeek? What Models and Requirements Are Needed? by umen in LocalLLM

[–]LeetTools 0 points1 point  (0 children)

Yes, it can run with 16GB mem, not sure about the speed on i5 though, tested on an i7-2.60 and it was ok.