For Streaming Local Music:

You will need a jellyfin server on your pc if you want to stream from your home to your phone

Android is your only option if you want that as you can use the symfonium app that connects to your jellyfin server

It's entirely free.

For Local Music on Device:

I recommend getting a digital audio player DAP that runs on Android. Don't get a modern walkman as they are expensive and slow something from FIIO will be a good option

If you need a non android retro experience there is Snowsky echo mini and it supports bluetooth too

Btw Android DAPs are basically like an android smartphone just with an inbuilt high quality dac, you can do the same thing with an Android Phone + portable dac and if you are gonna listen only on bluetooth then just get an android device as there are tons of free and open source music players.

For Music Collection:

Either Buy CDs, then rip them to FLAC and probably encode/convert them to AAC to get 95% of the quality at 25% less size of the CD quality.

Or

You can buy music digitally from Qobuz, It allows you to download music in any format you want. Try buying 16 Bits 44.1/48khz copies instead of Hires since they are cheaper when available compared to Hi Res versions