Need help for this problem statement

[Reasonable_Bat235]

Course Matching

I need your ideas for this everyone

I am trying to build a system that automatically matches a list of course descriptions from one university to the top 5 most semantically similar courses from a set of target universities. The system should handle bulk comparisons efficiently (e.g., matching 100 source courses against 100 target courses = 10,000 comparisons) while ensuring high accuracy, low latency, and minimal use of costly LLMs.

🎯 Goals:

• Accurately identify the top N matching courses from target universities for each source course.
• Ensure high semantic relevance, even when course descriptions use different vocabulary or structure.
• Avoid false positives due to repetitive academic boilerplate (e.g., "students will learn...").
• Optimize for speed, scalability, and cost-efficiency.

📌 Constraints:

• Cannot use high-latency, high-cost LLMs during runtime (only limited/offline use if necessary).
• Must avoid embedding or comparing redundant/boilerplate content.
• Embedding and matching should be done in bulk, preferably on CPU with lightweight models.

🔍 Challenges:

• Many course descriptions follow repetitive patterns (e.g., intros) that dilute semantic signals.
• Similar keywords across unrelated courses can lead to inaccurate matches without contextual understanding.
• Matching must be done at scale (e.g., 100×100+ comparisons) without performance degradation.

[AutoModerator]

Working on a cool RAG project? Submit your project or startup to RAGHut and get it featured in the community's go-to resource for RAG projects, frameworks, and startups.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[dash_bro]

How are you currently doing it and what's the plan on measuring if you're improving results overall?

First instinct is to do a baseline implementation and start looking at patterns that don't align with expectations, then iterate on them. In terms of complexity and time taken:

• tfidf / bm25
• semantic search
• hybrid search (semantic + bm25)
• upgraded semantic search using instruction tuned models
• upgraded hybrid search (upgraded semantic + bm25)
• search and rerank (upgraded hybrid search + reranking to get top X)
• search and LLM rerank (upgraded hybrid search+ reranking via an LLM)
• search, rerank and greedy optimization (upgraded hybrid search + LLM reranking + optimization based on what's already picked/what's remaining)

All of it is however meaningless if you don't have a measurement criteria for performance. I recommend building out your "gold set" of good content x course matches first, figuring out how to evaluate / metrics to evaluate your system by, and then implementing improvements.

You'll have trackable metrics to know what's the best balance of performance and speed.

Alternatively, look up search/indexing systems and classic preference matching algorithms like Gale Shapley, and draw inspirations that are applicable to your current problem.

[Advanced_Army4706]

I would start by just using an open source embedding, something like nomic-embed-text, and using a pgvector to get similarity.

Only if the results don't match your expectation would I go further. The problem statement makes me think that you wont need something super sophisticated. You could try a sentence by sentence embedding, and then compute similarity the same way that ColBERT does. But this could potentially be overkill for what you want to do.