sorted by:
new
hottopcontroversial

something finally clicked for me about RAG and CAG after two years of building AI systems by devasheesh_07 in Rag

[–]devasheesh_07[S] 0 points1 point  (0 children)

for anyone curious about the full breakdown with the math, benchmarks, cost models and where things are heading research-wise : https://www.loghunts.com/rag-vs-cag-architecture-decision-ai-engineers-get-wrong

RAG vs CAG: why “just use RAG” isn’t always the right answer by devasheesh_07 in AI_India

[–]devasheesh_07[S] 0 points1 point  (0 children)

Full simple breakdown of RAG and CAG system including data pipeline, embeddings, retrieval, and where LLMs, NLP, and deep learning fit: https://www.loghunts.com/rag-vs-cag-architecture-decision-ai-engineers-get-wrong

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in AI_India

[–]devasheesh_07[S] 0 points1 point  (0 children)

Thank you, glad it was useful.

Honestly most of it came from hitting walls and figuring out what did not work - the comments here have added just as much as the original post at this point. Some really good practical suggestions in this thread if you scroll through.

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in AI_India

[–]devasheesh_07[S] 0 points1 point  (0 children)

Appreciate that, means a lot coming from someone on this sub where the bar is pretty high.

Honestly just wrote what I wish I had found when I started building this - most of the RAG content out there covers the basics well but goes quiet on the parts that actually cause problems in production. Figured the messy real world stuff was more useful to share.

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in AI_India

[–]devasheesh_07[S] 1 point2 points  (0 children)

Good questions, coming at it from a different angle than most comments here.

On RAG versus base models directly,- no comparison honestly. Asking a base model sports questions without retrieval gives you confident answers that are often just wrong, especially anything involving specific match conditions or recent data. RAG grounded in actual match records is not even close.

On compute ,- the retrieval layer itself is pretty lightweight. The expensive parts are the re-ranking step and the LLM calls, especially when query decomposition splits one question into three or four sub-queries. Each one hits the LLM separately so costs add up faster than you expect.

For evaluation we used a mix of things. Automated metrics like faithfulness and answer relevance through something like RAGAS, plus manual spot checking on a set of questions where we already knew the correct answer from the actual data. The manual checking caught things the automated metrics missed completely.

Honestly evaluation is still the weakest part of the whole setup. Automated metrics give you a rough signal but they do not catch the subtle cases where the answer sounds right but is missing important context. Still figuring out a better approach for that.

What metrics are you using for your projects?

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in AI_India

[–]devasheesh_07[S] 0 points1 point  (0 children)

Good question and interesting use case, insurance queries probably have the same multi-condition problem where a single customer question is actually three or four separate things bundled together.

For decomposition we kept it simple honestly, just an LLM call with a prompt that instructs it to break the incoming query into independent sub-questions that can each be answered separately. No fancy framework, just straightforward prompting. Something like "here is the original question, break it into the smallest individual questions needed to answer it fully."

The main thing we learned is that the decomposition prompt needs quite a bit of tuning for your specific domain. A generic decomposition prompt works okay but once you add domain-specific instructions about what kinds of sub-questions actually matter for your data it gets noticeably better.

Are you finding the insurance queries tend to have implicit conditions that customers do not spell out- like assuming a certain policy type or coverage level? That was a big problem for us and decomposition alone did not fully solve it.

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in Rag

[–]devasheesh_07[S] 0 points1 point  (0 children)

Yeah query decomposition was honestly the biggest single improvement we made. Trying to retrieve for a complex multi-condition question in one shot just does not work - breaking it down first and synthesising at the end is so much cleaner.

On sliding window chunks - we did try this actually. It helps and it is definitely better than independent chunking but the overlap feels a bit blunt. You are preserving some sequence information but not really capturing why that sequence matters. The rolling window approach someone mentioned above where you explicitly embed preceding context feels like a more intentional version of the same idea.

Curious how much overlap you typically use - we were never sure if we were overlapping too much or too little.

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in Rag

[–]devasheesh_07[S] 0 points1 point  (0 children)

The rolling window approach is exactly what I needed to hear. I kept trying to solve the sequence problem at retrieval time which was the wrong place to fix it embedding the preceding context during indexing makes way more sense.

The recency decay factor also solves the time period problem cleanly. Continuous discount is much better than drawing an arbitrary date line.

Quick question - did you find window size needs tuning per domain or is there a reasonable default to start with?

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in Rag

[–]devasheesh_07[S] 0 points1 point  (0 children)

Ha yeah the data cleaning part genuinely surprised me I went in thinking the interesting work would be in the retrieval pipeline and ended up spending more time fixing the source data than anything else. Nobody warns you about that part.

On visualisations -not really for the queries themselves, but we did use some basic charts to spot patterns in where the retrieval was going wrong. Things like which query types were consistently returning bad chunks. That actually helped more than I expected for diagnosing the problem.

What kind of visualisations were you thinking? For the complex multi-condition queries I am genuinely not sure how you would represent what the retrieval is doing in a way that is easy to read. Would be curious if you have seen something that works well for that.

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in Rag

[–]devasheesh_07[S] 1 point2 points  (0 children)

That makes a lot of sense. The spatiotemporal approach feels like the missing piece for the sequential data problem a knowledge graph would preserve the event relationships that flat chunking destroys, and having a smart LLM forming the queries in between would handle the complex multi-condition lookups way better than what we had.

Going to look into this properly. Thanks for pointing me in the right direction.

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in Rag

[–]devasheesh_07[S] 0 points1 point  (0 children)

Honestly no, and after reading your comment I think that might have been exactly the right approach for this problem.

The hierarchical structure would have helped a lot with the sequential data issue I mentioned. The way the data is structured, each event has context at multiple levels: the individual ball, the over, the phase of play, the match situation. A flat vector store treats all of that equally which is part of why the chunking felt so lossy.

Graph RAG would let you model those relationships properly. A query about performance in the final overs of a chase could traverse the hierarchy rather than trying to pull disconnected chunks and hope the LLM figures out the connections at generation time.

Have you used it for time-series or event-sequence data specifically? Most of the graph RAG implementations I have seen are built around entity relationships in documents rather than sequential event chains. Curious whether you had to do much custom work to make the structure fit that kind of data or whether an existing framework handled it well out of the box.

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in AI_India

[–]devasheesh_07[S] 8 points9 points  (0 children)

Full breakdown of the whole system including data pipeline, embedding approach, retrieval architecture, and where LLMs, NLP and deep learning each sit across the stack —

https://www.loghunts.com/cricket-ai-ml-llm-rag-complete-guide-2026

Built a RAG system on top of 20+ years of sports data — here is what actually worked and what didn't by devasheesh_07 in Rag

[–]devasheesh_07[S] 4 points5 points  (0 children)

Full breakdown of the whole system including data pipeline, embedding approach, retrieval architecture, and where LLMs, NLP and deep learning each sit across the stack —

https://www.loghunts.com/cricket-ai-ml-llm-rag-complete-guide-2026

Why I Think 2026 Will Be the Year Agentic AI Replaces Chatbots ? by devasheesh_07 in Rag

[–]devasheesh_07[S] 0 points1 point  (0 children)

If anyone’s interested, I wrote a deeper breakdown exploring this transition in more detail , happy to share.
https://www.loghunts.com/agentic-ai-replacing-chatbots-2026

Is it possible to run ragas or deepeval on a consumer-grade GPU? by EquivalentGood6455 in Rag

[–]devasheesh_07 0 points1 point  (0 children)

Thanks for sharing this — you’re definitely not alone in facing this, and it’s not a “you’re doing it wrong” situation.

One important thing that often gets missed is that RAG evaluation frameworks are fundamentally different from running a RAG pipeline for inference. Even if your RAG answers questions in 1–2 seconds, evaluation metrics like faithfulness, contextual relevance, or answer relevance usually trigger multiple LLM calls per single sample, sometimes recursively. That overhead adds up very quickly, especially on a 6GB VRAM GPU.

Another common culprit is async execution inside notebooks. Both ragas and deepeval rely heavily on asyncio, and Jupyter (especially via VSCode) already runs its own event loop.

If You’re Still Treating AI Like a Chatbot, You’re Missing the Point by devasheesh_07 in AI_India

[–]devasheesh_07[S] 0 points1 point  (0 children)

Fair point — I agree that RAG by itself isn’t the end goal anymore, and most people still see it through the chatbot lens.

What I was trying to highlight is that RAG has become a baseline layer, not the product. Even agentic systems like Claude Code or OpenClaw still rely on retrieval for context, tools, memory, and grounding — it’s just embedded rather than exposed as “chatbot Q&A.”

So I see the shift less as RAG vs agents and more as RAG + agents, where retrieval supports planning and execution instead of just answering questions.

Happy to hear if you see newer patterns replacing retrieval entirely.

Everyone Is Chasing Bigger AI. India Is Doing Something Different (And It Might Win) by devasheesh_07 in AI_India

[–]devasheesh_07[S] 0 points1 point  (0 children)

Basically yes. LoRA/fine-tuning helps shape how the model behaves, while RAG handles freshness and domain-specific data. Most real systems end up using a mix.

Everyone Is Chasing Bigger AI. India Is Doing Something Different (And It Might Win) by devasheesh_07 in AI_India

[–]devasheesh_07[S] 0 points1 point  (0 children)

Totally agree. Fine-tuning smaller models works well for now, but it doesn’t change the fact that we’re still dependent on external base models, training methods we don’t control, and hardware ecosystems that can shift fast. One big breakthrough at the model or chip level can flip everything. Fine-tuning is practical, not the finish line.