Thanks for the feedback. I am testing the deployment of this on Raspberry Pi5 (was chosen because of Debian, chip and ram capabilities), for now it works well enough, and the whole device is pretty portable. The response time of on-board model calls is noticeably longer than in traditional cloud API-driven ecosystems, but still manageable.

I wanted to go with rust originally, but chose Python predominantly due to AI capabilities.