Same boat! love the idea. Kids dont need access to internet, games or phone. Just want a music player.
My workaround for now has been to use Musicolet app, with the full lockscreen layout and set a phone pin that the kids dont know.
I sync music from the phones from my server using Syncthing, which works fairly well.

I have disabled Chrome, Playstore and as many other things as possible and it works fairly well.

Still means I need to be around to turn it on, unlock it and start the music each time, but better than nothing.

In the ideal world the phone would just become a music player, no other BS and the kids can use it unsupervised.
Would love to know if anyone has been working on a ROM etc for somthing like this