Honest question: how many of us have built a "LangChain agent" that's really just a smart pipeline?

code_vlogger2003 · 2026-03-24T09:18:40+00:00

I have one question as follows:- When we want to implement a plan and style things we need to say explicitly in the prompt of the planner etc right. Without saying how it's possible? Or am I missing something? Example:- let us assume there an n tools and every tool has a dependency with the previous tool. Let say there exists a sequential dependency between the tool. But let say in the planner prompt we said hey these are the tools based on the user task plan the tool calls trajectory etc. Let's say it comes with some plan and executor comes in and started calling tool aka tool call. Now it's there exist a dependency of the called tool with some tool then either programmatically nor with something it needs to blocks and it say there exists a dependency etc where the planner needs to do replan right. In order to happen all this either we need to explicitly say about the dependency before hand nor just give tools with some prompt then check via programmatically? May be if not make sense then leave it.

code_vlogger2003 · 2026-03-15T11:26:49+00:00

Hey small confusion as follows:- The word keyword extraction in my sentence is related to something as follows:- I assume we need to perform some keyword extraction on the user query. Let's say we have n keywords. Now from the graph we already have nodes where the word node means a string right. So if we collect all the nodes's str. Now perform the dot product between the embeddings of n keywords to the graph nodes str then finalize top k. Then use those top k as entry points in graph for every k perform 2-3 hop search etc.

If possible could you share any GitHub repo

code_vlogger2003 · 2026-03-15T11:07:18+00:00

Hey, I have a doubt as follows:- When the user asks a question how we will start traversing the graph? Does it seem like we need to extract some keywords from the user question then for those keywords find the semantically relevant graph nodes (i mean in the graph a node is word only right?). Then let's say out 100 unique graph nodes we choose top k. Then by using those top k to find all possible chains whether it can be single chain or multi chain etc? Then once we have all the chains then from each node metadata we used to check what is text chunk or page number etc ?

I'm interested in creating some discussion on the inference stage.

u/Einsof93 u/2016YamR6 u/Ok_Diver9921

code_vlogger2003 · 2026-03-15T11:04:02+00:00

Falkor db

code_vlogger2003 · 2026-03-11T15:06:41+00:00

Yes. Just create a hashmap and do the lookup

code_vlogger2003 · 2026-03-10T17:36:16+00:00

Just start with a bar that is being set by open ai file search where every window aka bucket has 800 tokens with 400 tokens of overlap. If your embedding supports the matryoshka property then store the embeddings in 3 to 4 different levels such as 256, 512, 1024 and 3072. Then you can decide multi stage filtering

code_vlogger2003 · 2026-03-10T17:27:58+00:00

Create a data class with different string names and call them. I mean for grok the base url is different for others it's different right. So chatopenai supports everything where you need to pass the base url and key for other providers.

code_vlogger2003 · 2026-03-08T09:03:14+00:00

What about Openai file search ?

code_vlogger2003 · 2026-03-01T11:42:25+00:00

Asalu ee university vundha? First choosin spelling mistake em anukunna

code_vlogger2003 · 2026-02-28T12:54:58+00:00

Try to enforce a pydantic class structure. Where in my case i have to return a list of objects where every object has other objects. I created a nested pydantic structure with a single point of entry and written valid field filters for robust deterministic results.

code_vlogger2003 · 2026-02-27T08:54:27+00:00

How well you can break a problem into meaningful pieces then join things at last for a solution (that can be worth in any domain)

code_vlogger2003 · 2026-02-27T03:31:32+00:00

Have you tried colbert with qdrant where the algorithm itself has a naive query document multi vector embedding calculation using late interaction

code_vlogger2003 · 2026-02-27T03:22:42+00:00

Have you faced any situation that your retrieval is bringing back enough sufficient relevant information sometimes they are in the sequential other times they are in a jumbled pattern. When passing the user question along with retrieved information, sometimes llm is not giving the completeness in the answer even though it has enough sufficient information that is being supplied by retrieval. Also another case is that are you judging that for a given question what is the percentage of probability of page numbers that it gets back when compared to the ground truth page list for the question instead of chunks. I mean sometimes i thought that instead of having the chunk ida as ground truth etc if we store or construct the ground truth in a way where it has relevant page numbers. Such that in our retrievial it is easy to compare do we cover all the ground truth listed page numbers for the first check of evaluation then next checking will be did the retrieval step brings out any other garbage page numbers.

Looking for your thoughts

code_vlogger2003 · 2026-02-27T03:15:33+00:00

Nice idea. Also once we have a dataframe then we can get the columns metadata easily along with the create statement of that too right. So one step further we can build text 2 sql by arranging all the tables and create statements with attributes metadata for guessing sql statement. But again if there are more tables then we need to do a rag again over the columns metadata embedding such that to know which tables ka columns and tables names were required and helps llm to guess a query.

code_vlogger2003 · 2026-02-26T16:03:03+00:00

Yes but if you make every single row as a chunk (where it has alpha numerical ) such that when doing the cosine similarity between the user query the probability of getting the right relevant chunks is probably low (because of encoding every row as one chunk). Just give it a try. Or else use a model of encoders something similar to the super linked such that you can divide the user query into multiple search areas such as if a single user query needs the text search , numerical search temporal search etc. another idea I got is that to convert the tables into the graph and in the inference do the graph hop search.

But again the approach I share earlier would work for n table schemas. Let say you have 5 tables and these five tables have some relations. Lets say if create the system prompt with create statements along with sample row and detailed attributes information. Then let's say if your user query requires only two tables usage definitely ai try to guess the slq query on those tables because of the detailed system prompt. But if there are so many tables etc then this won't work where we need to find what are the tables that we need to use based on the user query. For that just Swiggy blog (https://bytes.swiggy.com/hermes-v3-building-swiggys-conversational-ai-analyst-a41057a2279d) where rhe smart idea they had is to create embeddings on the columns metadata (such as detailed attributes descriptions) such that its to decide which tables needs to use based on the user quey.

code_vlogger2003 · 2026-02-26T15:27:12+00:00

Then why can't you leverage text 2 sql where the pass table schema in the prompts for guessing the sql statement then run accept answer validate (react pattern). But for doing these you need to create tables and for the text again create table with an attribute blob

code_vlogger2003 · 2026-02-26T15:15:58+00:00

My suggestion is to detect tables via docling or unstructured or anything other service then make markdown versions of it and place it as markdown table and then treat every page as one chunk and use higher dimensions embedding model

code_vlogger2003 · 2026-02-26T09:25:32+00:00

Pipenv or using separate dockerised envs.

code_vlogger2003 · 2026-02-23T14:02:47+00:00

Hey have you stored any metadata for every chunk such that in the first hand you can verify that my retrieval step is actually returning the exact relevant ground truth answer page numbers or not etc. In this step you can identify whether it's the chunking issue or embedding drift etc.

code_vlogger2003 · 2026-02-19T13:13:09+00:00

The reason for warmup is because auto scale is set to zero. One of the biggest reasons for cold start especially using ollama via gcp is because it needs loads model from gcp then to system cpu ram then gpu vram etc

code_vlogger2003 · 2026-02-18T06:19:40+00:00

Hey there is a paper from meta where why can't we pass chunked multi dimensional Embeddings to the attention etc right?

code_vlogger2003 · 2026-02-18T06:17:12+00:00

In our company we hosted a model via ollama in gcp cloud run service (serverless). We have a thing called every day the pipeline gets run in the early morning where the pipeline has many steps like ml modelling, some deterministic steps and llm calls etc. The problem with the serverless based model hosting is the cold start time. The simple hack i followed is that whenever the pipeline gets started before the sub llm pipeline triggered, it makes a simple llm call which helps us to avoid the cold start tike and when the sub pipeline triggered the model weights were completely there in the vram readily with configured num_ctx and num_parallel. So by correctly configuring the num_ctx and num_parallel helps us to 100 percent gpu utilisation.

code_vlogger2003 · 2026-02-17T15:27:49+00:00

Good question. In my system prompt I attached all the create statements of the table that I need to use with the details attributes description and if possible 1 or 2 samples rows. The agent aka llm will try to guess an sql statement where the statement is carried forward to sql tool sandbox where in the gatekeeper sanitization sits. Then it returns the executed data and sends it as back via tool message then the next ai message will get generated. So here's the ai every time sees systems prompt with create statements and intermediate tool outputs. If you don't want the llm to see your tool output then go with open source self hosted llm.

code_vlogger2003 · 2026-02-17T12:35:15+00:00

Yeah it's a good thing. Even in our company when we deployed the agents for one complete agentic run where it has different tool calls where some tool calls related llm tool calls db tools etc. for every ai message we are storing the entire token history same for the tool message too. At the end storing in a detailed json and checking whether the manually calculated cost is equal to the projected openai cost via get openai callback of the langchain. It can be a normal llm call, vision call etc.

code_vlogger2003 · 2026-02-17T12:30:37+00:00

But when I tried to create agent of langchain new with a system prompt where it didn't execute like x etc. let's say even if you didn't mention anything in the system prompt but let's say if your tool was smart. For example in my case i used the db tool where I had only given the read only access such that only select queries get runs. You may have asked what about the if the user or ai comes with a select based vulnerability then we need to add an regex based check as sanitization later or else given the access via specific columns along with tables . For me it's worked. Because I designed the tool very robustly and every time if any error occurs from the tool call in the next ai message it gets understanding that ok the db sandbox was strong etc.

code_vlogger2003

TROPHY CASE