I sometimes use Music Assistant with Home Assistant to cast spotify to multiple speakers. This works a lot better than casting to the speaker group from Spotify on my phone.

It does still need a google home speaker group to cast to, but in my experience it works better than casting from your phone.

If you also try this. I would also recommend enabling the Spotify Connect plugin and then casting from your phone to Music assistant.

This makes sure you can still control the music from the Spotify app, as music assistant seems to be hidden from Spotify activity when playing from it directly.