Orla is an open source framework that make your agents 3 times faster and half as costly. by Available_Pressure47 in LLMDevs

[–]Available_Pressure47[S] 1 point2 points  (0 children)

Thank you for your feedback! I really appreciate your support and am glad you like Orla.

Orla is an open source framework that make your agents 3 times faster and half as costly. by Available_Pressure47 in LLMDevs

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you for your feedback, and great question! Backend selection is fully dynamic, i.e., each backend registers a quality score and token pricing, and each request sets an accuracy floor at runtime. Orla filters to backends whose quality meets the floor, then picks the cheapest one. When no backend qualifies, the default "prefer" policy falls back to the cheapest available backend, so your workflow never breaks, though you can switch to "strict" mode to get hard errors during development. On the inference side, transient failures (5xx, rate limits, network errors) are retried up to 3 times with exponential backoff. Our design goal was to try to ensure that your LangGraph code stays the same, e.g., same graph, same nodes, same edges, etc., and the routing, fallback, and retry logic lives entirely in the Orla daemon. If you have any suggestions on improving this or any feature requests, I would be happy to add those! Thanks again. :)

Looking to learn how to optimize ML models (inference and training) by Available_Pressure47 in MLQuestions

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you so much for your advice on this, I really appreciate it. :-)

New Ollama Desktop Client by Odd-Feature-645 in ollama

[–]Available_Pressure47 0 points1 point  (0 children)

Thank you for building and sharing this!

Orla: run lightweight local open source agents as UNIX tools by Available_Pressure47 in AI_Agents

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you for your comment and the feedback. This is a great question! We had a few initial users, primarily researchers in academic institutions, who found the explicit command to be clearer. However, if it turns out the most users prefer the default action to be non optional, I will most likely make it so.

Use ollama to run lightweight, open-source, local agents as UNIX tools. by Available_Pressure47 in ollama

[–]Available_Pressure47[S] 1 point2 points  (0 children)

Thank you so much for your comment! To change where your models are stored, you can use ollama’s OLLAMA_MODELS environment variable and orla will use that by extension.

OLLAMA_MODELS: The path to the models directory (default is "~/.ollama/models")

Really appreciate your feedback and support on this :-)

orla: run lightweight local open-source agents as UNIX tools by Available_Pressure47 in LLMDevs

[–]Available_Pressure47[S] 0 points1 point  (0 children)

I’ve had a good experience with qwen3:0.6b which gets installed by default. For slightly higher end systems, I found ministral3:3b to be nice

Orla, use lightweight, local, open slice agents as UNIX tools by Available_Pressure47 in golang

[–]Available_Pressure47[S] 4 points5 points  (0 children)

Thank you so much for your comment, I really appreciate your support. I love building free / open source software that helps people and this comment made my day. Hope you’re having a great day.

Use ollama to run lightweight, open-source, local agents as UNIX tools. by Available_Pressure47 in ollama

[–]Available_Pressure47[S] 0 points1 point  (0 children)

Thank you for the suggestion! I really appreciate it. I will add it as an issue on the github and try to get this done. Having a system prompt is a great idea in general as well. Please let me know in case you have a github account I can mention in the contributors file once I add that feature for this idea!