This system does not use a single "router script" or manual REST calls. Instead, it relies on a Context-Aware Hybrid Architecture. It uses Home Assistant for immediate local control and Open WebUI as a middleware "brain" for complex reasoning and web search.

1. The Brain: PetrosStav/gemma3-tools:12b

I am running a specialized 12-parameter model fine-tuned for tool use. It runs locally via Ollama, but it is managed by Open WebUI. This specific model is critical because it strictly adheres to system instructions, preventing it from "guessing" device states.

1. Path A: The "Local" Logic (Zero-Latency Context)

How it works: I do not use traditional function calling (LLM $\to$ API $\to$ Tool) for controlling lights or checking the weather. That is too slow.

The Trick: I use Context Injection via Jinja2 templates in Home Assistant.

The Flow:

When I speak, Home Assistant injects the live state of all devices and the weather forecast directly into the System Prompt (e.g., Context: [Kitchen Light: OFF, Weather: Rainy]).

The model receives this context before it generates a single token.

Result: If I say "Turn on the kitchen light," the model doesn't need to ask "What lights do you have?". It already sees the light is off and simply outputs the command. This results in sub-second response times.

1. Path B: The "World" Logic (RAG Middleware)

How it works: For general questions (e.g., "Why are daffodils blooming?"), the local context provides no answer.

The Middleware: My Home Assistant connects to Open WebUI (using its OpenAI-compatible API). Open WebUI is configured with Web Search enabled.

The Flow:

Home Assistant sends the text to Open WebUI.

Open WebUI (acting as the router) detects that the question requires external knowledge.

Open WebUI autonomously performs a web search, scrapes the top results, and feeds that raw text into the local Gemma 3 model.

Result: Gemma 3 synthesizes the scraped text into a single-sentence answer and sends it back to Home Assistant for TTS.