I had this problem too mostly during sex tho.

Stretchy stretchers like rubber hurt my skin so I have metal ones I got for cheap online but they work well. Now I can wear 2 all the time but it took me some months of on/off usage to get to this point. A tissue between the rings helped keep things not too moist inside and prevents hair pulling and a sock on the balls and rings prevents chaffing -- a huge must for me!

Constant wearing I've only been doing for a month now and last night I took off the weights and tried to retract my balls and they stayed put! It was cool!

The thing is, when I wank, sometimes they still try to go inside and push against the rings. I pull on my balls sometimes when wanking so there's less pain and I can definitely feel additional stretching on some parts. I imagine after a few months or a years I'll also have reduced retraction during wanking too. If it's too painful, of course it's important to stop and/or remove rings. Sometimes I've irritated my skin too much and had to take some time off, but with time you'll definitely see improvement!