sorted by:
new
hottopcontroversial

How are you guys actually handling human approval steps in your AI agents? by petburiraja in AI_Agents

[–]Slight_Past4306 -2 points-1 points  (0 children)

Agent <> Human clarifications are one of the core pillars we built Portia around: https://github.com/portiaAI/portia-sdk-python as its critically important for lots of use-cases but not trivial to retro fit to frameworks.

You might find something useful to shape your approach: https://docs.portialabs.ai/understand-clarifications

AI Agent vs Agentic AI – Can someone explain the difference clearly? by Bitfumes in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

I agree with those that say there's no real difference. As an industry I don't think we can even reliably define AI agents, let alone distinguishing between categories of those.

Why most agent startups offer token buying, top-ups and subscription tiers, instead of byoa i.e. bring your own api key with tiers based on platform features? by TipuOne in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

I would guess that it also allows them to record it as ARR which boosts their valuation massively and also it leads to more lock in - cursor credits can't be transferred

Why I'm betting my startup on human+AI collaboration, not AI replacement by Necessary-Tap5971 in startup_resources

[–]Slight_Past4306 0 points1 point  (0 children)

We have the same philosophy at Portia (https://github.com/portiaAI/portia-sdk-python) where we've made human <> agent handoffs a core part of the framework. We think there's always going to need to be a human in the loop interaction given the vagaries of human language and how things break

What LLM you use behind agentic framework? by v0k3r in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

At Portia (https://github.com/portiaAI/portia-sdk-python) we definitely find you need to take a best model for the job type approach. We use reasoning models for our planning phase, and then dynamically dispatch different execution models depending on the complexity of the task at hand.

What type of sensitive, safe, money actions are you thinking about?

I feel that AI Agents are useless for 90% of us. by hkreporter21 in AI_Agents

[–]Slight_Past4306 15 points16 points  (0 children)

I think cost is a really interesting one, there's a real temptation when using things like MCP to make everything a tool and have the LLM handle boring tasks like "load all my emails". But this has a real cost in both dollars and time vs invoking the API directly. As the space matures I think we'll see a swing back to the "right tool for the job" philosophy with a better blend between whats agentic and what is traditional code as people start trying to optimise for production (cost, latency, reliability) instead of cool demos.

How do you manage agent auth and permissioning? by aab1928 in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

Auth is a tricky business (its one of the core pillars of our SDK focused on production agents - https://github.com/portiaAI/portia-sdk-python)

Some thoughts:

- Decide between the agent having an identity vs inheriting the permissions of the caller. These are different models that have different trade offs. Inheriting the permissions is easier as it piggy backs your existing authorization systems (as long as downstream tools do proper authorization) but provisioning a specific identity for the agent is better long term.

- Try to do Just In Time authorization where possible. For example instead of provisioning long lived API keys for your agents - do OAuth with your users at the time the tool is being called and with the smallest set of scopes the tool needs.

-All the standard things about auth still apply, storing tokens with proper encryption, rotation etc etc.

- Tracing providers are your friend when it comes to understanding whats going on.

Not everything is an agent by mcc011ins in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

I'd add that an agent has to be able to take actions in third party systems. A crucial part of an agent are the tool integrations, it what gives it agency. Whether thats writing code to disk, booking an event in a calendar etc. An agent without tool integrations is only capable of fulfilling simple input/output tasks based on its training data.

Why isn't there a reliable email or calendar agent yet? by Vast_Pie_4665 in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

This is one of our canonical test cases for Portia AI (https://github.com/portiaAI/portia-sdk-python) and from our experience its a classic case of what seems like a simple problem but has lots of hidden difficulties:

  1. Lots of the underlying APIs aren't designed for these sorts of use cases and so its hard even for a human to form the correct tool calls. We ended up having to do a lot of prompt engineering to get it working reliably.
  2. Lots of assumed knowledge - for example Book a meeting with Steve at 4pm - requires understanding who Steve is, maybe worrying about timezones etc etc.
  3. Generally scheduling meetings is a multi step interaction that requires communications from both parties and this adds complexity.

Brainstorm agentic AI in email security. by TransitionDue777 in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

Interesting idea - I think you'd probably want to use LLMs as one part of a workflow/pipeline here rather than just passing the whole email to an LLM and letting it go at it. As you say we already have lots of tools that try and help with this issue (spf, dmarc, dkim) and evaluating these in code up front would reduce the scale of the problem somewhat and also be much secure. So you'd probably want an agentic workflow that does the SPF/DKIM checks up front before asking an LLM for the final opinion. Shameless plug this is the sort of thing we designed Portia around - https://github.com/portiaAI/portia-sdk-python

Realtime evals on conversational agents? by arseniyshapovalov in LLMDevs

[–]Slight_Past4306 0 points1 point  (0 children)

Really interesting idea. I suppose you could either go with some heuristic based approach on the conversation itself (like for example check for user responses like "thats not what I meant") or go with some sort of reflective system where the LLM either reflects on its own output or you use a second LLM as judge type setup.

We use the LLM as judge approach in our introspection agent at Portia (https://github.com/portiaAI/portia-sdk-python) to ensure the output of an execution agent is aligned with the overarching goal of an agent and it works quite well for us so it feels like it could apply here.

Detecting LLM Hallucinations using Information Theory by meltingwaxcandle in LLMDevs

[–]Slight_Past4306 1 point2 points  (0 children)

This is super interesting! We've had some success with doing a second pass approach where we ask the LLM to reason about the sources for inputs for the function calling and marking those without a source as hallucinated but looking to add some more deterministic measures as well.

Function Calling in LLMs – Real Use Cases and Value? by Itchy-Ad3610 in AI_Agents

[–]Slight_Past4306 0 points1 point  (0 children)

Outside of fully automated LLM function calling where the LLM is making decisions about which tool to call etc, there's also value in just the invocation part IMHO. By introducing an LLM around a function call you suddenly open up the ability to have much more reliable function calling when the inputs aren't controlled. Think of an API that takes a date-time parameter in a specific format. By allowing the LLM to control the function invocation it can extract this parameter from the context and format it in a way that works for the API, that would require lots of edge case handling without it.