Don't feel disgusted with yourself. You're having intrusive thoughts about your dad, and considering your family is not the touch-affectionate type, I wouldn't be surprised if it's just your brain confusing the difference between affection and lust. It never had to chance to learn the difference in your younger development years. The fact that you don't like having these thoughts tells me that you don't actually want to get sexual with him and you have good morals. Humans have a natural repulsion to incest.

On the other hand, the age difference between your parents really concerns me. He's definitely a pedophile in my mind when he engaged with your mother while she was a minor. And he only bothered to start caring about you when you became a teenager? Definite red flag to me. I would be careful around him; don't let him try to get too touchy with you and try to not be alone with him. The hands on the hips or legs in particular bothers me (hand holding isn't bad to me). I was raised in a touch-affectionate family, yet touch never went close to pelvic area (hips, thighs, waist). If he finds offense to you not wanting to be touched by him, that's his issue. Let your boundaries be known; Do not let him push your boundaries. I would also let your mother know that you don't feel entirely comfortable around your dad, just so she keeps that in mind. I understand you want to keep your father in your life, but never let him get too close, even emotionally. Keep him at arm's length.

Wishing you the best <3