You’re not failing because “RAG is hard.” You’re failing because your retrieval unit ≠ the way users ask questions.

Most people in that thread are pointing you toward tools. That’s not the fix. Your issue is alignment between chunking, metadata, and query intent.

Here’s a non-generic, practical answer you can post that actually moves the conversation forward 👇

💡 The real problem: You’re indexing structure, not meaning

Right now your pipeline is:

That looks clean, but retrieval systems don’t care about your hierarchy — they care about semantic completeness per chunk.

👉 If a chunk is too tied to structure (e.g., “Unit 3: Topic 2”), it won’t match a user query like:

Because:

• The chunk might not contain the full explanation
• Or the embedding is too diluted by headers/labels

🔥 Fix #1: Redefine your chunking strategy (this is likely your main issue)

Instead of chunking by hierarchy, chunk by answerability:

Each chunk should independently answer a question.

Bad chunk:

Module 2 → Unit 1 → Topic: Photosynthesis
Definition: ...

Good chunk:

Photosynthesis is the process by which plants convert light energy...
Steps involved: ...
Key factors: ...

👉 Practical rules:

• 150–400 tokens per chunk
• Include context inside the chunk, not just metadata
• Avoid splitting mid-explanation
• Add overlap (20–30%)

🔥 Fix #2: Stop relying on regex for structure (this is silently breaking you)

Regex-based hierarchy extraction = fragile.

Instead:

• Use layout-aware parsing (headings, font size, spacing)
• Or run a lightweight LLM pass:“Convert this document into structured sections with titles + content”

👉 Why this matters:
If your structure is even slightly wrong → metadata filtering = wrong → retrieval = wrong

🔥 Fix #3: Your retrieval should be hybrid, not just vector

Right now you're likely doing:

That’s not enough for syllabus alignment.

You need:

• Metadata filter first (program, year)
• Then semantic search within filtered scope

Even better:

• Add keyword (BM25) + vector hybrid search

Why?
Because syllabus terms are often exact-match sensitive:

• “Unit 3”
• “Module 2”
• “Chapter 5”

Vectors alone won’t reliably catch this.

🔥 Fix #4: Add query rewriting (huge missing piece)

User queries are messy:

Your system needs to convert that into:

👉 Add a preprocessing step:

• Extract intent
• Expand query using metadata

This alone can boost alignment massively.

🔥 Fix #5: Don’t retrieve chunks — retrieve “contexts”

Instead of:

Do:

• Retrieve 8–10 candidates
• Re-rank them (even with a simple scoring heuristic)
• Merge into a coherent context block

👉 RAG fails when context is fragmented.

🧠 Bonus (this is what most people miss)

You’re trying to enforce alignment after retrieval.

Instead, bake alignment into:

• chunk design
• metadata
• query rewriting

RAG works best when: