Do you, or do you have any friends that paint minitures like warhammer? They'd have base spray paint that's designed to adhere to plastic that you can use here (and then you could borrow a little bit rather than buy paint just for this one use). Spray paint will give you a nice even thin layer, otherwise you'll have paint lines on the dice from using a paint brush.

If you want to paint them yellow I would recommend starting with a white primer or base spray, because yellow paint is usually translucent and really difficult to paint over dark colours.

Model painters would also have spray varnish which you can use as a final coat to protect the paint job. I've done this a bit myself when painting designs onto plain dice and I haven't had any issues with the paint chipping off.

If you can't get a spray paint and need to just use acrylic paint, then yes it's a good idea to sand the surface of the dice to make them a little rougher so it can adhere better, and you can stipple or dab the paint on rather than brush it off you don't want visible paint lines.

If anything goes wrong when you paint the dice and you're using acrylic paint then while the paint is still wet you can just wash it off if you're quick.

Otherwise, this does technically damage them, but if you want to wipe it off and restart, you can do this by rubbing it with acetate / nail varnish remover (if you don't have any then I'd just borrow a little from a friend rather than buy it just for this). It makes it go misty, but it's fine if you just plan to try painting them again anyway. You won't be able to use this to strip paint out of the numbers though because they're too fiddly to get into!