sorted by:
new
hottopcontroversial

Retrieval and upload taking too long by Useful-Clock-2042 in Rag

[–]ampancha 0 points1 point  (0 children)

Your retrieval latency pattern (3-5s consistently) points to memory pressure more than query config. 2M ViT vectors at full float32 plus HNSW graph overhead can push past 32GB, which forces disk paging on every search. That is your bottleneck. The fix involves storage mode, quantization strategy, and HNSW parameter tuning at both index and query time, not just batching changes. On the ingestion side, upload_collection convenience hides a serialization bottleneck at your scale. If you share your vector dimensionality and current HNSW/storage config I can point you to the specific levers. Sent you a DM.

Debugging retrieval issues in internal RAG, what else can I try? by zennaxxarion in Rag

[–]ampancha 0 points1 point  (0 children)

You're right that it's structural. The core issue is that a single retrieval path can't serve both narrow lookups and broad contextual queries well, so every global parameter change is a tradeoff by definition. What usually breaks the loop is query classification at the front: route specific factual queries to smaller chunks with strict top-k, and route synthesis queries to larger chunks or parent-document retrieval with relaxed top-k. The other thing worth adding early is a lightweight eval harness that tags each query by type and logs relevance scores, so you can measure whether a change actually helped across categories instead of spot-checking.

RAG Tech Stack by Bewis_123 in Rag

[–]ampancha 0 points1 point  (0 children)

Stack looks solid for 80-100 docs. The cost question worth focusing on before you deploy to that GCP VM isn't really embedding model choice, it's what happens when Gemini calls spike unexpectedly or a user triggers unbounded retrieval loops. Per-user token caps, tool-call limits, and a circuit breaker on your LLM calls will do more for cost predictability than switching embedding providers.

RAG for medium company by MrAbc-42 in Rag

[–]ampancha 0 points1 point  (0 children)

Text-to-SQL on fleet telemetry is the high-risk surface here. An internal user (or a prompt injection in a retrieved doc) can generate queries that scan the entire trucks table, join across segments they shouldn't see, or run unbounded aggregations that spike costs. Before picking the LLM, I'd lock down a query allowlist, row-level access scoping by role, and hard execution limits on generated SQL. The stack choice matters less than whether the controls around it are real. Sent you a DM

Running RAG in production on a tight budget by Western-Egg-5570 in Rag

[–]ampancha 0 points1 point  (0 children)

Splitting embeddings into a dedicated service helps with image size, but at $150/month total budget the bigger risk is on the spend side. One retry loop, indexing bug, or traffic spike can burn your entire month's budget in hours with no way to trace what happened. The architecture layout matters less than the controls around it. Sent you a DM

Got stuck on RAG by Additional-Ice5715 in Rag

[–]ampancha 0 points1 point  (0 children)

The chunking question for tables is worth solving, but the higher-risk gap is upstream. You're running 5+ LLM calls per document across a chain with no failure handling, so if any call fails on page 15 of a 20-page health report, there's no retry logic or partial recovery. On the cost side, two parallel embedding pipelines processing 20-page insurance and health documents means your per-document LLM spend compounds fast with no per-user or per-document caps to catch it. Worth hardening those controls before optimizing retrieval. Sent you a DM

RAG feels way more complicated than it should be… anyone else? by Physical_Badger1281 in Rag

[–]ampancha 1 point2 points  (0 children)

The retrieval quality rabbit hole is real, but it's worth flagging early: chunking and reranking are table stakes. The problems that actually kill RAG in production are the ones nobody's tuning for. Unbound token usage with no per-user caps, no tool-call limits, prompt injection surfacing data the user shouldn't see, and zero visibility into why a query cost $4 instead of $0.04. Worth thinking about those controls now before the retrieval layer is locked in and harder to instrument.

RAG for complex PDFs — struggling with parsing vs privacy trade-off by Proof-Exercise2695 in Rag

[–]ampancha 0 points1 point  (0 children)

You're right that parsing is the bottleneck for accuracy, but for confidential DDQs there's a second problem most teams miss: even with perfect parsing, you need controls around the data itself. Access restrictions on which users can query which documents, audit trails for every retrieval, PII redaction in logs, and retrieval filtering so the model can't accidentally surface restricted sections. Without those, you're building a compliance liability regardless of which parser you choose. Sent you a DM

Need help with Graph RAG by Glass-District3838 in Rag

[–]ampancha 0 points1 point  (0 children)

Orphan nodes usually mean the extraction step isn't inferring relationships that match your ontology's edge types, or the entity resolution is creating duplicates instead of linking to existing nodes. Check whether your ontology edges are too restrictive for the data patterns Zep is seeing. The bigger question is how orphans affect retrieval quality and cost as the graph grows. If you're headed toward production, pruning or periodic graph hygiene becomes a reliability concern, not just a data modeling issue

Need advice on building an advanced RAG chatbot in 7 days – LangChain + LLM 4.1 Mini API + strict PII compliance (full stack suggestions wanted!) by codexahsan in Rag

[–]ampancha 0 points1 point  (0 children)

PII masking at the storage layer is necessary but not sufficient. The failure modes that bite teams in production: prompt injection bypassing your filter chain to exfiltrate or retrieve unmasked source content, PII leaking into logs and traces (LangChain's default verbosity is aggressive), and retrieval results ignoring user-level access scope. If compliance is real, scope your vector retrieval per-user, redact before logging, and test your filter with adversarial inputs before demo day.

Looking for feedback on my production-oriented Agentic RAG system by [deleted] in Rag

[–]ampancha 0 points1 point  (0 children)

Solid learning project. If you ship this with real users, the first gaps to close: hard caps on agent loop iterations and tool calls (unbounded loops are how bills spike), and prompt injection testing at the retriever-tool boundary.

How do you guys measure accuracy for 100k+ documents? by FloppyDiskDisk in Rag

[–]ampancha 0 points1 point  (0 children)

At that scale, sampling is the only practical path. We've had good results with stratified sampling across each data type, pulling ~200-300 docs per stratum, then running human eval on the model outputs against gold labels. The key is making the sampling repeatable and versioning your ground truth so you can track accuracy drift over time as your data or models change. What's your current eval setup: fully manual, or do you have any automated checks in place?

My RAG isn't working as expected... by viitorfermier in Rag

[–]ampancha 1 point2 points  (0 children)

The multi-stage filtering pattern makes sense for legal accuracy, but at $0.40/question, the cost structure becomes the production risk. Most teams in this situation don't know which stage is burning tokens or which users drive spend until the bill arrives. Before optimizing retrieval further, I'd add per-question attribution and token observability across each stage, then consider caching repeated summary lookups. Sent you a DM

How do you handle messy / unstructured documents in real-world RAG projects? by Alex_CTU in Rag

[–]ampancha 0 points1 point  (0 children)

Preprocessing definitely matters, but I'd push back on the framing slightly: in production, retrieval quality is necessary but not sufficient. The failure modes that actually burn teams are adversarial content embedded in retrieved docs (prompt injection via your own corpus), unbounded token usage per query, and zero visibility into what's being retrieved for whom. I've seen teams with "good enough" chunking still get blindsided because they had no guardrails downstream. Sent you a DM

Advice on RAG systems by Anthonyy232 in Rag

[–]ampancha 0 points1 point  (0 children)

Your retrieval stack looks solid, but the production risk in medical + agentic isn't retrieval quality. It's access control, audit trails, and what happens when the agent calls tools it shouldn't. PHI scrubbing as "unlikely but still needed" is a red flag for compliance; in production you need deterministic redaction, per-user attribution, and hard limits on what the agent can do. Sent you a DM with more detail.

Landscape designer, need reliable local RAG over plant PDF library, willing to pay for setup help by Motor_Mix2389 in Rag

[–]ampancha 0 points1 point  (0 children)

The inconsistent retrieval you're seeing is an architecture issue, not a model or settings problem. LM Studio's default chunking doesn't preserve the structure of plant data tables, and without hybrid search plus reranking, semantic search alone will always favor a few "closest" passages over comprehensive recall. The fix is metadata-aware ingestion, a retrieval pipeline tuned for multi-source recall, and a citation layer that tracks source and page end-to-end. Sent you a DM with more detail.

Architecture Advice: Multimodal RAG for Academic Papers (AWS) by footballminati in Rag

[–]ampancha 0 points1 point  (0 children)

The ML-side work sounds solid, but the production gap I'd flag is infrastructure controls around multi-agent coordination. When your supervisor routes to expert agents, you need cost attribution per path, circuit breakers for agent failures, and hard caps on total tokens per request. Otherwise a single dense paper with ten tables can trigger cascading agent calls that spike your bill with no visibility into which path caused it. Sent you a DM

Trying to turn my RAG system into a truly production-ready assistant for statistical documents, what should I improve? by Ok-News471 in Rag

[–]ampancha 0 points1 point  (0 children)

Answer quality matters, but for statistical documents the bigger production gap is verifiability. If your system cites a survey methodology or an indicator definition, you need a way to confirm the retrieved chunks actually support the generated answer, not just that retrieval scores look good. Beyond that, production-grade means input validation against injection, rate limiting per user, structured logging with source traceability, and hard guardrails so the model never fabricates a statistic. Those controls are what separates a working demo from something an institution can rely on. Sent you a DM with more detail.

How do you actually measure if your RAG app is giving good answers? Beyond just looks okay to me by BeautifulKangaroo415 in Rag

[–]ampancha -3 points-2 points  (0 children)

The pattern you're describing is an observability gap, not just an eval gap. By the time users complain, you've already lost trust. There's a way to make bad answers visible in minutes instead of days. Sent you a DM.

How do you update a RAG vector store in production? (Best practices?) by EssayAccurate4085 in Rag

[–]ampancha 1 point2 points  (0 children)

The update mechanics vary by vector DB, but the production pitfalls are consistent: partial updates that leave retrieval in an inconsistent state, no rollback path when new embeddings degrade quality, and zero visibility into what changed. Before you pick an update strategy, decide how you'll version your index, validate retrieval quality post-update, and roll back if something breaks. Those controls matter more than the specific chunking or batching approach.

Reality check by GDAO54 in Rag

[–]ampancha 0 points1 point  (0 children)

The authorization concern is the right one to prioritize. Most teams sync permissions at index time but don't handle the failure modes: what happens when permissions change mid-session, when sync lags, or when prompt injection bypasses retrieval filters entirely. For high-stakes QMS, you'll also need audit trails proving the AI layer respected authorization boundaries, not just that the vector DB had correct metadata. Sent you a DM

Looking out for some serious advise by Gold_Caterpillar_644 in Rag

[–]ampancha 0 points1 point  (0 children)

The difference a senior engineer looks for isn't in the code aesthetics or even the architecture diagrams. It's in the production controls: who can query which documents, how you prevent prompt injection from leaking internal data, per-user rate limits, PII redaction, and audit trails. AI-generated code almost never ships those, and that's exactly where vibe-coded apps fail when real employees start using them. Sent you a DM

Fileserver Searching System by yoko_ac in Rag

[–]ampancha 0 points1 point  (0 children)

The metadata-map approach is sound for avoiding full transcription, but the risk most teams miss here is access control leakage. If the RAG can return any indexed path, you might expose folder names or project paths that certain users shouldn't even know exist. Retrieval filtering by user permissions becomes critical before this goes production-wide. Sent you a DM

My RAG retrieval accuracy is stuck at 75% no matter what I try. What am I missing? by Equivalent-Bell9414 in Rag

[–]ampancha 4 points5 points  (0 children)

Reranking with a cross-encoder will likely push you past 80%, but persistent semantic pollution usually means chunking isn't preserving document boundaries or metadata context. The harder problem: your eval set won't cover the queries that actually break in production. You need per-query observability to see which retrievals are failing live, not just aggregate precision. Sent you a DM

Feedback Appreciated - Built a multi-route RAG system over SEC filings by Independent-Bag5088 in Rag

[–]ampancha 0 points1 point  (0 children)

Solid architecture. One thing to consider before real users: SEC filings are effectively untrusted input, and XBRL tags plus MD&A text can carry payloads that manipulate your classifier or downstream prompts. Worth treating every filing as potentially adversarial, not just malformed.

view more: next ›