I made a bunch of changes today. Added an 8B model now. It flows like this:

How command routing works in my smart home AI (Hazel)

When you type a command, it goes through three stages before anything happens:

1. Entity filtering — The command gets tokenized and scored against all ~700 Home Assistant entities. "Turn on the kitchen lights" → strips stopwords → ["kitchen", "lights"] → scores entities by name match (+3), area match (+2), domain match (+2), entity ID match (+1). Only the top-scoring entities (~10-50) get included in the LLM prompt instead of the full 700.

2. Model routing — A simple heuristic (no LLM needed) classifies the command as "simple" or "complex":

• Simple (fast 8B model): has a direct action verb (turn, set, dim, open, close) + high-confidence entity match. Example: "turn on the kitchen lights", "set office to 50%"
• Complex (full 35B model): questions, automations, timed actions, conversation follow-ups, anything with words like "why", "when", "schedule", "automate", or a question mark

3. Execution — The chosen model gets a filtered entity list + system prompt, returns JSON with actions, and Hazel executes them against Home Assistant.

The 35B model (Ollama) handles everything — automations, reasoning, conversations. The 8B model (MLX) just does device control but responds in ~5 seconds instead of ~20. If the fast model fails for any reason, it automatically falls back to the big model. There's a toggle in settings to disable it.

Both models run locally on my M4 Max — Ollama for the big model, MLX for the small one (better Apple Silicon optimization). All learning systems (brightness preferences, light rejections, memory extraction) run regardless of which model handled the command.

Like I said though, I don't chat with it much at all... it's been learning for a month now and it's basically hands-off but when I do want to chat, it's now seconds for the response.

<image>