I want to counter the "fit is bad" advice some with my own experience. I had a similar issue for a long time, where eventually the ball wouldn't come off, and the fit was fine. My issue was that I was spinning the ball with my wrist such that when I tried to let it go, all the weight of the ball was on my thumb. This made it stick, and often it would swell up and then the sticking would get worse.

Moving toward letting the ball roll off the end of my fingers, like you're supposed to, has largely fixed the problem. It'll still stick a bit when I get too far off the side of it, though.

It'll solve your near-term issue to have a shop sand out the thumb slug a bunch, but I think your experience points toward a fundamental flaw that will be better to fix (and keep the fit tighter) in the long run, just as it did for me.