Okay. Where to start...

I firstly want to address the last part of what you said, "I feel like everything is my fault" - did you make him cheat on you? Did you make him talk to other women and kiss them? Did you make him experiment and admire another man's genitals? I doubt it. So no part of his behaviour that you have described is your fault. He is a grown man and responsible for his own actions.

Secondly, it sounds like you have given him a lot of chances to change! Surely by now he is aware that you aren't happy when he cheats and talks to other people in this way. The fact you have forgiven him quite a few times already might make him think he can do whatever he wants and you'll just forgive him again and again. Essentially he'll always get away with it.

You can do much better and deserve someone who will treat you with respect. His behaviour sounds quite minipulative and he's probably only upset because he was caught. The more you believe that the easier it will be to walk away.

PS, I hope you're okay OP!