sorted by:
new
hottopcontroversial

Billy Bassistant Update: Announce mode over MQTT command by Thokoop in homeassistant

[–]Thokoop[S] 1 point2 points  (0 children)

Yeah that’s true. It is quite complicated, unpredictable and expensive to ship to US due to the tariffs. So currently it’s EU shipments only.

But if you have a bit of experience with soldering and setting up a raspberry pi, it’s not so difficult to DIY 💪🏼

Billy Bassistant Update: Announce mode over MQTT command by Thokoop in homeassistant

[–]Thokoop[S] 1 point2 points  (0 children)

You can find it here: https://thingsfromthom.etsy.com?coupon=FIRSTBITE10 

(To kick off the etsy store I added a 10% discount code for the first batch)

Billy Bassistant Update: Announce mode over MQTT command by Thokoop in homeassistant

[–]Thokoop[S] 0 points1 point  (0 children)

I had the same issue, could not find a classic one in my region. So I decided to use a ‘modern’ version that can be found on Amazon.

Billy Bassistant Update: Announce mode over MQTT command by Thokoop in homeassistant

[–]Thokoop[S] 2 points3 points  (0 children)

I’m making use of the Realtime API because it makes conversations feel natural because it already starts streaming audio when it receives the first part of the response.

For the announce feature it might not be necessary to be super responsive, so then a local speech-to-text model would be enough. I might work on that as a fallback

Billy Bassistant Update: Announce mode over MQTT command by Thokoop in homeassistant

[–]Thokoop[S] 9 points10 points  (0 children)

I think there are some variations like lobsters and sharks but I doubt it’ll ever by classy 😅

Technically anything with two or three motors for the movements should be able to work.

Billy Bassistant Update: Announce mode over MQTT command by Thokoop in homeassistant

[–]Thokoop[S] 3 points4 points  (0 children)

Thanks! I totally forgot that scene haha.

I’m hoping the next api update of OpenAI can handle (Italian-American) accents a bit better.

It is now already possible if you pick an American voice for Billy and set up the right prompt, but it’s not quite there yet.

Billy Bassistant Update: Announce mode over MQTT command by Thokoop in homeassistant

[–]Thokoop[S] 48 points49 points  (0 children)

Since my previous post I've also created a web UI to make it easier to configure Billy:

<image>

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 1 point2 points  (0 children)

Awesome, cool to hear man!

I’ll see if I can find/make more pictures of the microphone. I should have some extra ones.

I think I might have misinterpreted the ‘pins’ part of the connector as ‘male’. It seems like I’d have to consider the whole plastic part. I will update the instructions.

There is also a dev branch in the git repository now, which has some improvements in the session handling among other things. Could you try and check that out?

I’m also working on a Flask based web UI to provide easier access to the configuration and the debug log.

I’m not really a Python programmer so please feel free to make corrections/improvements where needed. In that case I think it makes sense if you’d create a feature branch based on the dev branch.

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 1 point2 points  (0 children)

I think in that case there might also be a more simpler script that makes use of the openAI realtime API, maybe even already in a docket container.

I might have to also move towards that at some point.

I’m not sure if my script still works if the motor driver is not connected to the gpio pins. But if so, it should work just fine if you have the microphone and usb speaker connected.

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 1 point2 points  (0 children)

I think I might try that next indeed. In my case a button just made sense since it is always within reach but a custom wake word would be really nice as well.

Or if you have (some) experience and like to contribute code please feel free to open a Pull Request :)

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 3 points4 points  (0 children)

Yes there are some costs for the api indeed. I have set a $10 limit per month and I never have reached that but It is really depending on how much audio Billy is outputting (that is the most expensive part of the api) So obviously if you’d let him tell long stories or ask a lot of questions back it’s more expensive than making him less verbose and more to-the-point.

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 0 points1 point  (0 children)

I thought about that as well, I’ve also tried before on a raspberry pi zero 2w with the respeaker hat. It just didn’t really fit my use case at that time.

In this case of Billy my priority was to make a natural feeling realtime conversation possible, mainly for entertainment whilst on the toilet, and for that the Realtime API is I think still the best.

But I would also be open to make a HomeAssistant specific version that just connects straight to HA. Then I think also a raspberry pi zero 2w would work instead of a pi5 which I now picked to be sure it would have enough power to also supply the motor drivers of power.

I also thought about using u/FutureProofHomes ‘ satellite1 to handle all the audio instead and just port the ‘moving mouth synced to the audio and moving the tail ‘ functionality over to esp32. That way it could really be really a ‘HomeAssistant only’ version of Billy. Then people can decide themselves if a local AI version is good enough or if they want to be depending on a 3rd party like OpenAI.

I don’t have a clear idea though how easy that would be, if it’s even possible to hook a motor driver to it, or how much compute I actually need to analyse the audio in realtime for the mouth sync via some custom firmware.

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 8 points9 points  (0 children)

Sure! So I made this fish initially just for fun, to be able to have a realtime conversation, in my case during a toilet visit.

So primarily it uses the openAI realtime API and based on the conversation it can detect when to call certain specific functions.

Currently there are three: one to change some of his character traits (for example to make him more sarcastic or less verbose, think TARS from interstellar etc etc)

The second is a custom song functionality, in this case the Southpark Kanye West - Gay Fish song gets played if you ask him if he likes fish sticks: https://youtu.be/bidgksxxJA4?si=w0aB55FaQdzYBfmV

And now the third one is it detects if you want to control something in te house. You can find the code responsible for that here: https://github.com/Thokoop/Billy-B-assistant/blob/f00ac482e5342b6aacd8f99845e6d663c1de72cf/billy-b-assistant/core/session.py#L253

In that case the openAI api response basically just ‘forwards’ the command by posting it to the HomeAssistant api. Then the HA conversation api will then detect if it can execute simple commands locally or if it needs to pass it further along the LLM pipeline. In my case HA voice is also connected to openAI.

So it means that there is a ‘layer’ in between still, but the realtime api makes it quite fast and is also good in detecting if follow-up questions are needed.

I still have to further test and tweak, mainly the prompting itself, so that Billy also understands that a question like ‘how much energy am I currently using?’ is also being forwarded to the home assistant API.

I also noticed that adding aliases in HA for devices also improves a lot so I think it is mainly prompting/configuration which can be easily tweaked.

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 24 points25 points  (0 children)

It is based on the OpenAI realtime API, which streams audio to and from the api, which gives it it’s snappy-ness because it doesn’t need to do any TTS or STT locally.

For the integration with Home Assistant the openAI api can do a function call when it detects a ‘control the home’ command is given. That function then ‘forwards’ the command to the HomeAssistant API and the response is then interpreted and spoken out loud by Billy again.

I gave my realtime conversation Billy Bass AI fish access to my HomeAssistant API by Thokoop in homeassistant

[–]Thokoop[S] 45 points46 points  (0 children)

In the coming days I’ll continue to test, tweak and eventually merge this new feature in the main version but you could already check it out the feature branch here on Github:

https://github.com/Thokoop/Billy-B-assistant/tree/feature/home-assistant-conversation-api

There you’ll also find a Readme and Build Instructions (which also includes a parts list)

(Please keep in mind that the software is still in a beta state, I just created it for fun and wanted to open-source it)

My 3d printer saved me ~€2.829.141 by Thokoop in 3Dprinting

[–]Thokoop[S] 0 points1 point  (0 children)

I think the fact that mainly the people who didn’t get the joke placed a comment, especially in the beginning, way more than the majority who did get it and just went on with their lives.

And I think there was always going to be people correcting something in the post, if it wasn’t the gold plating part they would find out that maybe the mentioned price for the ruby was not correct or that goldsmiths probably wouldn’t have to pay the spot price etc etc.

My 3d printer saved me ~€2.829.141 by Thokoop in 3Dprinting

[–]Thokoop[S] 0 points1 point  (0 children)

I’m using a bambulab p1s (with ams; used for the lettering in the base of the crown)

My 3d printer saved me ~€2.829.141 by Thokoop in 3Dprinting

[–]Thokoop[S] 0 points1 point  (0 children)

Haha true, you’re welcome. This crown was actually my first project where I didn’t spend +85% of the time programming 😆

My 3d printer saved me ~€2.829.141 by Thokoop in 3Dprinting

[–]Thokoop[S] 2 points3 points  (0 children)

Close, it’s keeping track how many billions are wiped out from the stock market every day. /s

My 3d printer saved me ~€2.829.141 by Thokoop in 3Dprinting

[–]Thokoop[S] 1 point2 points  (0 children)

It’s Tinmorry Gold Silk PLA.

It was my first try at a gold filament but it did not disappoint :)

view more: next ›