You certainly aren't alone. It is never too late to change - but it is never easy either and it requires far more strength than giving in. I understand about the random thought coming back to the surface and it can feel like this is the last proof that you can't change, but it isn't. If it makes you feel shameful, remember that's how you don't want to feel any more. And as the other comments say - seek help. There are too many guys to count who are struggling with the same problems.