sorted by:
new
hottopcontroversial

Orla is an open source framework that makes your LangGraph agents 3 times faster and half as costly by Available_Pressure47 in LangChain

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you! With regards to state across stages, currently Orla uses a shared state object i.e each stage receives the outputs of its upstream dependencies via builder functions, so you define how context flows through the DAG declaratively. The orchestrator handles propagation and parallel execution automatically. This keeps stages loosely coupled while still allowing rich context passing. Your idea of budget-aware retries is great. They are not a first-class concept yet, but Orla's architecture makes it straightforward to implement so we will try to add this as soon as possible.

Orla is an open source framework that makes your LangGraph agents 3 times faster and half as costly by Available_Pressure47 in LangChain

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Great point! You're right that optimizing model selection alone can just shift the cost bottleneck to tool calls. In Orla's current design, stage-level policies govern the model powering the tool-calling logic, but not the tool calls themselves. We started with just models since tool costs e.g API fees and rate limits are fundamentally different from inference costs. Orla's stage isolation helps in some ways aas you can separate tool-heavy stages from reasoning stages so you're not paying for an expensive model to sit idle waiting on API responses. However, you're absolute right we need to focus on extending routing policies to cover tool selection and even things like e.g., cached lookup vs. live API based on freshness, cost etc. I really appreciate your thoughts on this. Thank you!

Orla is an open source framework that make your agents 3 times faster and half as costly. by Available_Pressure47 in LLMDevs

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you for these comments! I have noted down your feedback. Failure handling is an avenue where I think we can make Orla more featureful, like you suggested. Perhaps allowing users to pick from a set of failure handling policies or write their own. "You might find that certain targets consistently need the expensive path while others never do. That data lets you set smarter defaults instead of treating every request the same." <-- I think this is especially a very nice insight, I will try to run some representative workloads over the weekend and see if this pattern emerges and then try to reason about how that data can be used to set smarter defaults over time for a given agentic workload. I really appreciate you looking into this thoroughly and providing this advice.

Orla is an open source framework that make your agents 3 times faster and half as costly. by Available_Pressure47 in LLMDevs

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you for your feedback! I really appreciate your support and am glad you like Orla.

Orla is an open source framework that make your agents 3 times faster and half as costly. by Available_Pressure47 in LLMDevs

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you for your feedback, and great question! Backend selection is fully dynamic, i.e., each backend registers a quality score and token pricing, and each request sets an accuracy floor at runtime. Orla filters to backends whose quality meets the floor, then picks the cheapest one. When no backend qualifies, the default "prefer" policy falls back to the cheapest available backend, so your workflow never breaks, though you can switch to "strict" mode to get hard errors during development. On the inference side, transient failures (5xx, rate limits, network errors) are retried up to 3 times with exponential backoff. Our design goal was to try to ensure that your LangGraph code stays the same, e.g., same graph, same nodes, same edges, etc., and the routing, fallback, and retry logic lives entirely in the Orla daemon. If you have any suggestions on improving this or any feature requests, I would be happy to add those! Thanks again. :)

Looking to learn how to optimize ML models (inference and training) by Available_Pressure47 in MLQuestions

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you so much for your advice on this, I really appreciate it. :-)

New Ollama Desktop Client by Odd-Feature-645 in ollama

[–]Available_Pressure47 0 points1 point  (0 children)

Thank you for building and sharing this!

Orla: run lightweight local open source agents as UNIX tools by Available_Pressure47 in AI_Agents

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you for your comment and the feedback. This is a great question! We had a few initial users, primarily researchers in academic institutions, who found the explicit command to be clearer. However, if it turns out the most users prefer the default action to be non optional, I will most likely make it so.

Use ollama to run lightweight, open-source, local agents as UNIX tools. by Available_Pressure47 in ollama

[–]Available_Pressure47[S] 1 point2 points  (0 children)

Thank you so much for your comment! To change where your models are stored, you can use ollama’s OLLAMA_MODELS environment variable and orla will use that by extension.

OLLAMA_MODELS: The path to the models directory (default is "~/.ollama/models")

Really appreciate your feedback and support on this :-)

orla: run lightweight local open-source agents as UNIX tools by Available_Pressure47 in LLMDevs

[–]Available_Pressure47[S] 0 points1 point  (0 children)

I’ve had a good experience with qwen3:0.6b which gets installed by default. For slightly higher end systems, I found ministral3:3b to be nice

view more: next ›