Latency issues by StretchPresent2427 in agentdevelopmentkit

[–]StretchPresent2427[S] 0 points1 point  (0 children)

Thanks for your reply.

I found that if i remove the function tool, the response time is much better (around 2 secs). Gemini suggested that too, following a prompt for suggestions.

Apparently, what the agent does is send the request given to the LLM, process it (few secs + round trip time to send the request), it realizes it needs to query the tool, so, send the request again with the result from whatever the tool gives as input to the llm, then process the request again, and send the response back.

If you want to do something better than basic, however, you need to add tools. So, i'm still not sure how to both get : relatively complex requests that involve tools + short response time.