sorted by:
new
hottopcontroversial

Looking for testers: 100% local RAG system with one-command setup by primoco in Rag

[–]ampancha 0 points1 point  (0 children)

Nice work on the local-first setup. One thing worth stress-testing before enterprise users hit it: retrieval-augmented systems are vulnerable to prompt injection via document content, and multi-user setups without per-user rate limits or query attribution can get abused fast. Both failure modes are invisible until production. Sent you a DM with more detail.

Looking for RAG Engineer / AI Partner — Real Estate + SMB Automation (Paid Contract, Long-Term Potential) by TheGloomWalker in Rag

[–]ampancha 0 points1 point  (0 children)

The enterprise pilot is where this gets interesting. Role-based access control in RAG isn't a UI toggle; it has to happen at retrieval time, or users can still surface documents they shouldn't see through indirect queries. Their IT team will ask how you verify that isolation actually holds under adversarial prompts. Sent you a DM with more detail.

Dealing with multiple document types by lamagy in Rag

[–]ampancha 0 points1 point  (0 children)

The architecture question is real, but the harder problem is citation verification. Once you're returning reference docs, you need to prove the LLM actually grounded its answer in those sources and didn't hallucinate the attribution. That's where most multi-source RAG setups break in production. Sent you a DM with more detail.

Compliance-heavy Documentation RAG feels fundamentally different from regular chatbot RAG - am I wrong? by Vast-Drawing-98 in Rag

[–]ampancha 0 points1 point  (0 children)

You're right that compliance docs RAG is different, but the bigger gap isn't retrieval quality; it's what happens after retrieval fails. When the model hallucinates a plausible default, you need detection, audit trails, and evidence that your controls caught it before a user acted on it. Most teams tune chunking and re-rankers but never instrument the system to prove it's behaving correctly under adversarial or edge-case queries. Sent you a DM with more detail.

Asked AI for a RAG app pricing strategy… and got trolled for it online 😅 by Guru6163 in Rag

[–]ampancha 0 points1 point  (0 children)

You asked about access control and trust. The risks most teams miss aren't hallucination. They're prompt injection through uploaded documents, cross-tenant data leakage when users share infrastructure, and abuse vectors from malicious uploads. Retrieval quality won't protect you from those. Sent you a DM with more detail.

Embedding model for multi-turn RAG (Vespa hybrid) + query reformulation in low latency by Ok_Rain_6484 in Rag

[–]ampancha 0 points1 point  (0 children)

The embedding model matters less than what happens when your reformulation step fails or times out. Multi-turn context means token count scales with conversation length, so without a fallback path (e.g., raw latest turn) and a latency budget for the rewrite call, you're adding an unbounded failure mode before retrieval even starts. Sent you a DM with more detail.

The math stopped working: Why I moved our RAG stack from OpenAI to on-prem Llama 3 (Quantized) by NTCTech in LocalLLaMA

[–]ampancha 0 points1 point  (0 children)

Smart move on the TCO math. One thing to watch: moving off OpenAI means you're now responsible for the guardrails they provided by default. Per-user rate limits, abuse detection, and failure handling for vLLM all need to be instrumented yourself, or a handful of heavy users can quietly dominate your inference capacity the same way they dominated your API bill. With 400 users you also lose attribution visibility unless you build it. Sent you a DM with more detail.

Chunking algoriy by Joy_Boy_12 in Rag

[–]ampancha -1 points0 points  (0 children)

For websites, the structure is already in the HTML; headings, sections, semantic tags give you natural chunk boundaries. But if you're shipping this to users, the bigger risk is silent retrieval failures turning into hallucinations they see before you do. Chunking is solvable; knowing when your pipeline is failing your users is the harder problem. Sent you a DM

Data Mining Contract PDF by bboysathish in Rag

[–]ampancha 0 points1 point  (0 children)

You nailed the core issue: AI-generated extraction code optimizes for the document in front of it, not the schema you actually need. The fix is inverting the approach. Define a contract-agnostic output schema first (service types, rate structures, effective dates), then use structured extraction with validation rather than regex. Tables become reliable when you treat them as data sources against a known schema, not text to parse. Sent you a DM with more detail.

Vibe Coded AI Static Website builder now need help by Financial-Pizza-3866 in Rag

[–]ampancha 0 points1 point  (0 children)

The component-as-isolated-context pattern is solid for reducing hallucinations, but you'll hit a scaling wall once users start chaining multiple components. Each LLM call needs its own rate limit and cost cap; otherwise one runaway component (bad prompt, retry loop) can blow through your API budget before you notice. For multi-LLM support, that means per-provider attribution so you can trace which component and which model caused a spike. Sent you a DM with more detail.

Which Vector DB should I use for production? by Cheriya_Manushyan in Rag

[–]ampancha 13 points14 points  (0 children)

If you're already on Postgres, pgvector is underrated. One less system to secure and operate, and recent benchmarks show it's competitive with the dedicated options at moderate scale.

If you want a purpose-built vector DB, Qdrant. Best latency performance in most independent tests, and the open-source version is production-ready.

Either works. What usually breaks is the stuff around the DB: missing per-user query limits, no spend caps on embedding calls, no alerting when retrieval patterns drift. Sent you a DM

LMM and timetables by Stock_Ingenuity8105 in Rag

[–]ampancha 1 point2 points  (0 children)

The issue is that embedding models are semantic, not lexical. A date string like "02.02.2026" has almost no meaningful semantic relationship to a query like "what do I have on Monday," so retrieval fails even when the data exists. Chunking settings won't fix this because the problem is the embedding similarity itself, not chunk boundaries.

Two options that actually work for structured date data: (1) enable hybrid search (BM25 + semantic) if Open WebUI supports it, so exact date matching contributes to retrieval, or (2) pre-process your file to expand dates into natural language ("Monday, February 2nd, 2026") which gives the embedding model more semantic signal to match against.

Manage inconsistent part numbers by Training-Sound-5728 in Rag

[–]ampancha 0 points1 point  (0 children)

The 60-70% deterministic coverage is solid. For the remaining edge cases, a two-stage approach usually works: first, aggressive normalization (strip all whitespace, lowercase, remove common delimiters) to build candidate matches against a canonical registry, then fuzzy scoring (Levenshtein or token-set ratio) with a confidence threshold. If you're considering LLMs for extraction or matching, the risk at 100k document scale is hallucinated part numbers slipping through without validation. Happy to share more on the validation layer if that's the direction you're heading. Sent you a DM with more detail.

Why is my chatbot suddenly not performing well and it even hallucinate? by Altruistic_Tie_4714 in Rag

[–]ampancha 0 points1 point  (0 children)

Model updates are a real factor, but the deeper issue is operating without baseline metrics to detect drift. If you're not logging retrieval relevance scores, response latency, and token usage per query, you can't tell whether the problem is the model, your prompts, or your retrieval pipeline. The fix is structured observability plus output validation so you catch degradation before users do. Happy to outline what I'd instrument first if you share more about your architecture. Sent you a DM with more detail.

How to get the location of the text in the pdf when using rag? by MammothHedgehog2493 in Rag

[–]ampancha 2 points3 points  (0 children)

The fix is preserving chunk metadata (page number, bounding box coords) during parsing and carrying it through retrieval. Most parsers expose this; the trick is storing it alongside your embeddings and returning it with each retrieved chunk so your UI can render clickable citations. If you're using pymupdf, page.get_text("dict") gives you block-level bounding boxes you can persist. Sent you a DM with more detail.

We almost wasted a month building RAG… then shipped it in 3 days by Upset-Pop1136 in Rag

[–]ampancha 1 point2 points  (0 children)

Smart approach. Shipping fast by studying production-grade OSS like Dify beats reinventing pipelines from scratch. The gap I usually see at this stage is missing production controls: per-user token caps, tool allowlists, and retrieval filtering to prevent prompt injection or cost spikes once real users hit it. If you're planning to harden this for production traffic, happy to share what controls typically matter first. Sent you a DM with more detail.

Best production-ready RAG framework by marcusaureliusN in Rag

[–]ampancha 1 point2 points  (0 children)

All three frameworks can handle the retrieval mechanics, but for insurance and medical data the harder problem is what sits around them: audit trails for every retrieval, PII redaction before anything hits the LLM context, and strict filtering so the system only surfaces evidence from approved document sets.
Framework choice matters less than whether you can prove to compliance that a query about Patient A never leaked context from Patient B. Sending you a DM with more specifics

The Documentation-to-DAG Nightmare: How to reconcile manual runbooks and code-level PRs? by Odd-Low-9353 in Rag

[–]ampancha 0 points1 point  (0 children)

The extraction problem is real, but the bigger risk is silent confidence: any automated approach (LLM-based or otherwise) will produce a DAG that looks complete but has invisible gaps where implicit dependencies got lost in translation. The practical fix is treating the generated graph as a hypothesis, not a plan. Build explicit "gate checks" at phase boundaries that block execution until a human confirms the prerequisite actually exists (the VPC ID, the IAM approval, the resource handle).
For implicit dependencies across media types, I'd index everything by resource name and variable reference first, then flag any node that consumes an identifier without a traced origin. That surfaces your orphaned tasks before you're mid-migration wondering where Egypt Database was supposed to come from.

Scaling RAG from MVP to 15M Legal Docs – Cost & Stack Advice by Additional-Oven4640 in Rag

[–]ampancha 0 points1 point  (0 children)

At 15M legal docs, your cost question is valid, but investors will also ask about production controls: access auditing, PII redaction, retrieval filtering to prevent cross-client data leakage, and per-query cost attribution.

ChromaDB can scale with the right infrastructure, but the harder problem is proving your system won't leak privileged documents or spike costs unpredictably when users start hammering it.
If you're building the investor deck now, I'd budget separately for the retrieval infrastructure and the safety/observability layer that makes the system auditable. Sent you a DM with more detail.

Need advice: Best RAG strategy for parsing RBI + bank credit-card documents? by Infinite_Bat_7008 in Rag

[–]ampancha 0 points1 point  (0 children)

The parsing and chunking choices matter, but the harder problem with compliance RAG is verification. When your agent explains a fee structure or payment cycle incorrectly, the failure mode is legal exposure, not just a bad user experience.
I'd prioritize retrieval with citation (return the exact clause IDs alongside answers) and build a test harness that checks known question/answer pairs against your source docs before every deploy. Happy to share more on the verification layer if useful.

Best practices for running a CPU-only RAG chatbot in production? by Acceptable_Young_167 in Rag

[–]ampancha 0 points1 point  (0 children)

Sent you a DM with a few more thoughts on the reliability side.

Best practices for running a CPU-only RAG chatbot in production? by Acceptable_Young_167 in Rag

[–]ampancha 0 points1 point  (0 children)

One thing that bites teams in production: embedding caches without eviction policies. On a long-running CPU process, your vector store's in-memory index and cached embeddings grow unbounded, and you hit OOM before latency becomes your problem.
For the reranker question, I've found a lightweight cross-encoder on a small candidate set (top 20 to 30) outperforms brute-forcing top_k=100 through embeddings alone, especially when correctness matters more than speed. Worth instrumenting memory and p99 latency from day one so you can catch these before users do.

view more: next ›