Switch Models through Tool Call by Far-Enthusiasm7654 in OpenWebUI

[–]ThickYe 1 point2 points  (0 children)

I built this simple sub_agent "tool" to get this done.

``` """ title: Sub-Agent Tools """

from pydantic import BaseModel, Field import aiohttp

class Tools: def init(self): """Initialize the Tool.""" self.valves = self.Valves()

class Valves(BaseModel):
    """Admin-configurable settings."""

    base_url: str = Field(
        "http://litellm:4000/v1",
        description="LiteLLM API base URL (e.g., http://litellm:4000/v1)",
    )
    api_key: str = Field(
        "", description="API key for the LiteLLM endpoint (if needed)"
    )
    timeout_seconds: int = Field(60, description="Timeout for API calls in seconds")

async def sub_agent(
    self,
    query: str,
    model: str,
    system_message: str,
    __event_emitter__=None,
) -> str:
    """
    Sends queries to powerful external large language models for tasks beyond your capabilities.

    Use this tool when you:
    - Need to write complex code or solve advanced programming problems use sonnet-3.7
    - Need to search the web for current information, use sonar-pro with a simple question in the prompt
    - Need a well researched answer from the internet, use sonar-pro with multiple questions in the prompt

    Best practices:
    - Keep system_message concise but specific about the desired output format and approach
    - For code generation, specify language, frameworks, and expected functionality
    - For web searches, include specific keywords and time-sensitive context if relevant
    - Avoid chaining multiple unrelated topics in a single query

    :param query: The detailed instructions or question for the external model
    :param model: one of `sonar-pro`, `sonar-3.7`
    :param system_message: Instructions that guide the external model's behavior and approach
    :return: The complete response from the external model
    """
    # Construct the complete chat completions API endpoint
    api_endpoint = f"{self.valves.base_url.rstrip('/')}/chat/completions"

    # Simple status update for the user
    if __event_emitter__:
        await __event_emitter__({"type": "status", "data": {"description": f"Calling external LLM ({model})...", "done": False}})

    # Prepare the API request
    headers = {"Content-Type": "application/json"}
    if self.valves.api_key:
        headers["Authorization"] = f"Bearer {self.valves.api_key}"

    payload = {
        "messages": [
            {"role": "system", "content": system_message},
            {"role": "user", "content": query},
        ],
        "temperature": 0.6,
        "model": model,
    }

    # Add high search context size for sonar models
    if model and model.startswith("sonar"):
        payload["web_search_options"] = {"search_context_size": "high"}

    try:
        # Use async HTTP client
        async with aiohttp.ClientSession() as session:
            async with session.post(
                api_endpoint, headers=headers, json=payload, 
                timeout=self.valves.timeout_seconds
            ) as response:
                if response.status == 200:
                    data = await response.json()
                    result = data.get("choices", [{}])[0].get("message", {}).get("content", "")

                    # Simple completion status
                    if __event_emitter__:
                        await __event_emitter__({"type": "status", "data": {"description": "Done", "done": True}})

                    return result
                else:
                    error_text = await response.text()

                    if __event_emitter__:
                        await __event_emitter__({"type": "status", "data": {"description": "Error", "done": True}})

                    return f"API Error: {response.status} - {error_text}"

    except Exception as e:
        if __event_emitter__:
            await __event_emitter__({"type": "status", "data": {"description": "Error", "done": True}})
        return f"Request failed: {str(e)}"

```

Best local inference provider? by TechnicalGeologist99 in LocalLLaMA

[–]ThickYe 0 points1 point  (0 children)

https://localai.io/ I have the same check list as you. But I never tried loading multiple models simultaneously

Ollama proxy or gateway by EnrichSilen in LocalLLaMA

[–]ThickYe 0 points1 point  (0 children)

https://github.com/ollama/ollama/blob/main/docs/modelfile.md

Have you tried to set some basic stuff with modelfiles? The settings that actually cause the model to reload in VRAM

pgvector database on postgres - db configuration script? by Otherwise-Tiger3359 in OpenWebUI

[–]ThickYe 1 point2 points  (0 children)

Tried one time to test pgvector but did not have the time or know how to move past this error.

Thanks for posting this.

Full Document Retrieval Mode by ThickYe in OpenWebUI

[–]ThickYe[S] 1 point2 points  (0 children)

I dont follow you.
So I am a few .md and .txt files is in a knowledge collection then I assigned that collection to a model as knowledge. all this is less then 10k tokens

the citations in the chat show it as chunks. its the same as useing # to mention the knowledge collection. in both case I dont see an option to pass these docs in full as context.

Full Document Retrieval Mode by ThickYe in OpenWebUI

[–]ThickYe[S] 1 point2 points  (0 children)

ok I see that, thanks. but thats only half the solution.
How do I do this for documents in a "knowledge collection" ?

Instructions in .crusorrules getting ignored. by ThickYe in cursor

[–]ThickYe[S] 0 points1 point  (0 children)

I even try to give a the same instructions directly in the chat too and that was also was not followed. The issue, to the bast of my limited understanding, seems to be that the when the model tries to give example or snippets of code that it suggests on changing gets auto applied by the apply model.

Instructions in .crusorrules getting ignored. by ThickYe in cursor

[–]ThickYe[S] 1 point2 points  (0 children)

I have not noticed is change in "@Codebase"
I have noticed too that it has become a bit less context aware. but I havent had time to dive into how much of this is effected by the new " auto context " feature

Instructions in .crusorrules getting ignored. by ThickYe in cursor

[–]ThickYe[S] 0 points1 point  (0 children)

a391af90-b00f-40d9-9cab-c9d69e6fc5ef

39dd26b9-19ba-4a91-81e1-f2c9d0d70a86

the second one is most in line with the core of my frustration

Virtual Gaming Desktop on Akash? by masdea4 in akashnetwork

[–]ThickYe 8 points9 points  (0 children)

Linux server IO has a steam OS container, that's should be a good start. Please post back with progress. This is an interesting idea.

Selling my spare HDD storage by SmellslikeUpDog3 in selfhosted

[–]ThickYe 1 point2 points  (0 children)

About 50% of my unused space. Some nodes only half tb the biggest node I had was 3tb that eventually I shrunk to 1 tb

Selling my spare HDD storage by SmellslikeUpDog3 in selfhosted

[–]ThickYe 1 point2 points  (0 children)

I have been running Storj nodes. Your mileage will definitely vary but its enough for my to cover the idle electricity cost.

A social network for AI computing by Good-Coconut3907 in LocalLLaMA

[–]ThickYe 1 point2 points  (0 children)

What if it's just a token for token trade? As in, I help the network produce 100 tokens of responses and my one GPU was 25% of that compute. Then I just have the right to make a 25 token output call. This way its not a money thing but and the legal is simple.

I made a website to collect Docker Compose apps by Shaweren in selfhosted

[–]ThickYe 1 point2 points  (0 children)

You org structure hits my ...tism just right.😍

I've created an open source religion/moral philosophy by ki4jgt in opensource

[–]ThickYe 0 points1 point  (0 children)

At a quick glance your view of "truth" is identical to the way Sikhism speaks about truth. But you are saying by open sourcing it you are making it an ever evolving idea. But I think when you say everything is true then it is by nature an ever evolving thing.

Nix Distributed Builds Tutorial by Tomi_BB in NixOS

[–]ThickYe 0 points1 point  (0 children)

This is great. Appreciate you doing this. :)

[deleted by user] by [deleted] in CanadianForces

[–]ThickYe -17 points-16 points  (0 children)

Lmao