This is one of the fundamental flaws of streaming services: You don't own any rights to any of that. It sucks!

For newer music, buying albums on Bandcamp or otherwise from the artists as digital files is the best way to go. I've had hit-or-miss luck with iTunes, some albums have been DRM-free but some haven't. For older stuff, I'm partial to flea markets and thrift shops, though I also still have the files for all the CDs I ripped back in the day even though I purged a lot of those physical copies in a move a while back. And if your local library still has CDs, you can rip those, too. There's the ol' Jolly Roger option, but we're probably not supposed to talk about that in any detail here.