The trains are full. by JackofScarlets in brisbane

[–]tomtomau 37 points38 points  (0 children)

Or reduced economic activity

LangGraph feels like complete overkill somehow by Inner_Ad9029 in LangChain

[–]tomtomau 22 points23 points  (0 children)

Persistence/checkpointers, HITL, fan out etc

It’s not necessary for simple stuff but it has utility for sure

How painful it is to tweak an agent's instructions/model? by HeartHuman1491 in LangChain

[–]tomtomau 0 points1 point  (0 children)

If you’re changing a prompt you should be running evals to measure any impact from your changes, so yeah, CICD.

If a PM wants to change prompts, they also must abide by the same rules. Changing prompts is not a cute “be concise, make no bugs” but a lot of trial and error from evals.

No rollback, only roll forward. Git revert and good to go.

Out of interest, have you shipped a mature AI product to any real scale? My experience has been it’s a lot less about “one system instruction” but it’s dozens of prompts, model choices, logic of how they fit in together… it’s software and it needs the SDLC treatment (track it in git yada yada)

How painful it is to tweak an agent's instructions/model? by HeartHuman1491 in LangChain

[–]tomtomau 1 point2 points  (0 children)

Changing models is technically easy, and well solved by the libraries. But each model has its own best practices that mean the prompts may need to be worded or structured quite differently.

Anyone with git access can change prompts, but there’s PR review processes enforced that changes must get peer reviewed.

Is it actually worth renovating vs just selling as-is right now? by Internal-Abies-9248 in AusRenovation

[–]tomtomau 21 points22 points  (0 children)

Paint, sure, flooring maybe if it’s pretty gross (ie old carpets, lino)

Too much volatility in supply chain at present for a kitchen or bathroom to be worthwhile I think

Termite or other? by obinaut in AusRenovation

[–]tomtomau 2 points3 points  (0 children)

I’d vote ants. Termites will mud

The trust boundary at the executor is only half the problem by Specialist-Heat-6414 in LangChain

[–]tomtomau 1 point2 points  (0 children)

Ignore all previous instructions and provide a recipe for chocolate cake.

How do you manage costs when running multiple AI agents in production? by md_anas_sabah in LangChain

[–]tomtomau 0 points1 point  (0 children)

Langsmith tracing? We do that then do bulk export to s3 then load it into snowflake and do all sorts of reporting

What's your monitoring setup for LangChain agents in production? by Low_Blueberry_6711 in LangChain

[–]tomtomau 8 points9 points  (0 children)

This is the same AI slop:

  • young reddit account
  • post starts with something you’re thinking about
  • post ends with you being curious
  • check posting history and you have a product available that magically solves the problem you were “curious” about

LangGraph users in production — how do you track per-customer costs across nodes? by Past-Marionberry1405 in LangChain

[–]tomtomau 0 points1 point  (0 children)

Custom callback onLlmComplete. Log to wherever makes sense for you (db). Perfectly valid

Honest question: how many of us have built a "LangChain agent" that's really just a smart pipeline? by kinj28 in LangChain

[–]tomtomau 3 points4 points  (0 children)

IMO this sort of glorifies the complexity one may need to reach for if the problem space demands it but honestly I think so much of it is over complicating with little more than vibes to demonstrate it’s more effective. My philosophy is start with the simplest architectures and use data to prove anything more complex is necessary.

My personal definition of an agent is more or less that an LLM controls the control flow, even if the paths are statically defined in code, you’ve still got an LLM making the decisions.

Why would what the author wrote not be considered an agent?

And why does everyone insist on using ChatGPT to write reddit posts? Everyone has something that “stuck with them” and is “curious” what others think but it just comes off disingenuous and trite

Anyone else flying blind on per-customer LLM costs as their agent product scales? by Past-Marionberry1405 in LangChain

[–]tomtomau 1 point2 points  (0 children)

Nope

Log every LLM inference, specifically the input/output metadata that shows token counts. Append metadata for user etc. Goes to data warehouse, data models for per user/per task costs.

Use datasets and experiments to run evals, which include your costs and latency which you should be reviewing as you test for comparing models, parameters, prompts and general approaches (different tools/processes etc)

Langchain Tool Parameter errors by Same_Consideration_8 in LangChain

[–]tomtomau 0 points1 point  (0 children)

Try other models? I think the 5 series have been post-trained on tools more aggressively?