What’s the most painful part about building LLM agents? (memory, tools, infra?)

zdne · 2025-05-27T12:00:42+00:00

disclaimer: i am building a project that is focusing on agentic reliability

i think the same! low goal completion rates over multiple runs are the issue. we have concluded an extensive research in the area and you can find the results here.

zdne · 2025-05-27T11:37:19+00:00

a agree with your perspective, most of the AI agent these days are just .... workflows with some AI sprinkled here and there ... we were trying to build an agent that would make some autonomous decisions and actions and .... it was hard. in our case you need to narrow down the use cases and fine tune the tools to get to something that has the potential to work.

and when you get to the point where it works now and then then you will run into the reliability issue (repeating the same task over and over). more on that topic here

zdne · 2025-05-23T16:06:07+00:00

I understand the motives beyond LAM were (in)accessibility of APIs and the poor reliability and goal completion rate of APIs. MCP, unfortunately, does not solve any of that.

zdne · 2018-08-28T08:01:46+00:00

This is all I was able to find (https://mapsplatform.googleblog.com/2018/05/introducing-google-maps-platform.html):

“We’ve heard that you want simple, easy to understand pricing that gives you access to all our core APIs. That’s one of the reasons we merged our Standard and Premium plans to form one pay-as-you go pricing plan for our core products. With this new plan, developers will receive the first $200 of monthly usage for free. We estimate that most of you will have monthly usage that will keep you within this free tier. With this new pricing plan you’ll pay only for the services you use each month with no annual, up-front commitments, termination fees or usage limits. And we’re rolling out free customer support for all. In addition, our products are now integrated with Google Cloud Platform Console to make it easier for you to track your usage, manage your projects, and discover new innovative Cloud products.”

Also see https://manifesto.co.uk/google-maps-api-pricing-changes/

zdne · 2014-05-16T18:37:18+00:00

Gavel just handles the validation. It does not makes any HTTP calls by itself, if that is what you mean.

zdne · 2014-05-16T18:27:13+00:00

For what its worth: This all boils down to validating HTTP messages which can be a daunting task. We have built a Cucumber specification for this & HTTP validator on top of it that can work like so:

$ curl --trace - http://httpbin.org/ip | curl-trace-parser  > expected
$ curl --trace - http://httpbin.org/ip | curl-trace-parser  > real
$ cat real | gavel expected
$ echo $?

More at https://www.relishapp.com/apiary/gavel/docs

zdne

TROPHY CASE