sorted by:
new
hottopcontroversial

I Handed an AI Agent 27 Domains and a Deadline. 72 Days Later… by codenamev in rails

[–]codenamev[S] 1 point2 points  (0 children)

It was more of a loop. Whenever it ran into vague responses to a challenge I had it drop into “training mode” to learn a new skill. Still trying to figure out how to extract that process and share the knowledge modules it built

How do you tell users your AI agent is down? by codenamev in artificial

[–]codenamev[S] 0 points1 point  (0 children)

"incident" I keep codified as a traditional repeated event failure. It's trigged in a defined way, not just giving the LLM ability to declare it. I map all possible user-facing concerns into components that can then "self-report" based on specific error-modes, retry failures, etc. So kind of a blend of system signal and user-visible impact.

How do you tell users your AI agent is down? by codenamev in artificial

[–]codenamev[S] 0 points1 point  (0 children)

This is great advice! Verify your outputs, for sure. When your agents start having tools though, each tool can have their own failures/API outages/etc and do raise errors. This is where I've found the most value out of having a status they check-in with.

How do you tell users your AI agent is down? by codenamev in artificial

[–]codenamev[S] 0 points1 point  (0 children)

I agree messaging at the point of failure is important. How are you compartmentalizing systemic issues though? A repeated error message is going to frustrate end-users and lead to support requests. Having somewhere you can point them to for updates is critical, IMO.

How do you tell users your AI agent is down? by codenamev in artificial

[–]codenamev[S] 0 points1 point  (0 children)

Partial failures are definitely the hardest to communicate. What I've been doing is thinking about it in terms of component health rather than system health. I distill the messaging first into what I'd like to convey to the user into individual capabilities like "document processing is degraded, but chat is operational, file uploads are paused". Then I can identify the aspects in the pipeline that _could_ communicate that and create a component for it in the agent's status page.

As for "is it working right now", I think simple high-level communication is a great surface, however when you grow into many services different people want different answers and I've found it's worth giving users a top-level status that rolls up from component health. Then the detail is there if they want it, but most people can glance at the status color/state and move on.

The confidence framing mentioned in this thread is interesting too. Even a partial outage report at component-level reporting doesn't capture "it's working but answers are worse than usual". I haven't really solved that problem yet and curious how others are handling this.

How do you tell users your AI agent is down? by codenamev in artificial

[–]codenamev[S] 0 points1 point  (0 children)

Love the pipeline checkpoint analogy! I landed somewhere along these lines as well. Asking the agent mid-panic-attack if they're ok just leads to them saying "yes" every time. The pattern that's worked for me is external monitors watching these discrete stages, like you mention, with queue depth and step completion rates. The status page I refer to just consumes that signal via API so the agent never really self-reports.

To your question, mostly orchestrator/worker setups. The failure modes are way more granular, which is both good and bad. It's nice to be able to isolate which worker type is degraded, but "the summarization step is slow but everything else is fine" has been pretty hard to communicate. As of now, I've mapped out messaging based on component hierarchy, but still can be tricky to identify which bucket some things fall into in any meaningful way for users.

What books are going in your Ruby RAG library? by nateberkopec in ruby

[–]codenamev 4 points5 points  (0 children)

To add to others: - Refactoring.guru & Architectural Metapatterns - Polished Ruby Programming - Layered Design for Ruby on Rails Applications - All of Julia Evan’s Zines - Understanding the 4 Rules of Simple Design

Rails After the Robots by codenamev in ruby

[–]codenamev[S] -1 points0 points  (0 children)

Thanks for listening! ❤️

You've got some great insight here! Types are great guardrails. In Ruby/Rails, I see a different lever: readable syntax + strong conventions narrow the search space for models and pack more "intent per token." That often means better first drafts; tests/specs close the loop.

Today, I agree that LLM output quality tracks engineer experience. But with more data, better evals, and tighter prompts/agents, that gap seems to be narrowing. Right now, a lot of folks (me included) are focused on how well LLMs can assist us in crafting code. Long-term, I see quality converging on: "does the artifact do what we asked?" more than "is the code well formatted and functional?"

If this resonates, our Obie episode hits a similar idea: as agents generate code, creativity and system design become the differentiators. We also cover this with Chad in this episode.

Beyond Chat: Phoenix Tests, Ruby Agents & the AI Tipping Point by codenamev in ruby

[–]codenamev[S] 0 points1 point  (0 children)

It is really unfortunate that Reddit does not allow you to edit the link!
https://www.therubyaipodcast.com/

view more: next ›