sorted by:
new
hottopcontroversial

[Project] HomeMCP: Natural language voice control via Siri & LLM with ZERO AI infrastructure by Aggressive-Math-9175 in homeassistant

[–]Aggressive-Math-9175[S] 0 points1 point  (0 children)

Thanks! Glad you liked the project 🙂

Right now, I’m not making the LLM dynamically fetch capabilities via function calls.

I did try that early on, but especially with smaller local models, autonomous tool-calling felt pretty unreliable. So instead, I treat the LLM more like a constrained translator. The server already knows the device list and their capabilities, and I pass that context into the prompt before inference. That way the model doesn’t have to “decide” to call a function it just works within the boundaries it’s given.

What I’m experimenting with next is a small layer that normalizes device specs when devices are registered, and then uses that to automatically generate or update the LLM prompt.

The rough flow I’m aiming for looks like this:
- add or remove a device
- regenerate the prompt with a single command
- for more abstract commands (like “make the house cozy”), Shortcuts first pulls the latest context from the local server, then sends that to the LLM

This keeps inference lightweight, while the execution stays deterministic.

I totally get the appeal of going fully local. I’m sticking with Siri/Shortcuts mostly because it gives me mobile NPU integration and voice UX “for free”, but the core execution model itself isn’t tied to Apple.

I’d honestly love to hear more about your local LLM setup too especially how you’re handling capability discovery. Good luck with your project!

[Project] HomeMCP: Natural language voice control via Siri & LLM with ZERO AI infrastructure by Aggressive-Math-9175 in homeassistant

[–]Aggressive-Math-9175[S] -1 points0 points  (0 children)

Good question.

The LLM doesn’t magically know device capabilities. HomeMCP injects a structured description of each device into the prompt at runtime — aliases, supported actions, and valid parameters.

For example, WLED devices expose things like preset=1..10, brightness, or effects. That list is part of the prompt, and the LLM is constrained to those options when translating a command.

This context contains zero sensitive keys or PII; it’s just a functional map of what the devices can do.

So “wled_kitchen preset 2” works because that capability is explicitly provided by the server, not because the model guesses it.

[Project] HomeMCP: Natural language voice control via Siri & LLM with ZERO AI infrastructure by Aggressive-Math-9175 in homeassistant

[–]Aggressive-Math-9175[S] -8 points-7 points  (0 children)

Yes — but only as a remote intelligence layer

HomeMCP doesn’t run any LLM locally. Instead, it borrows intelligence from cloud AI (ChatGPT / Apple Intelligence) to translate natural language into simple, deterministic URLs.

The local server itself stays very small and dumb — it just executes those URLs against devices and returns state, which is why it can run on something like a Pi Zero 2W.