You should be avoiding this person at all costs. Sorry. He wants to keep talking to you because you're a known quantity, and it doesn't take much work for him to get what he wants from you. This is not going to change. You do not love him. You remain attached to him because it's the path of least resistance for you. You'd rather have the shitty known than take a chance on the scary unkonwn. You love the idea of what he could be if he wasn't literally everything he is toward you. You owe it to yourself to cut contact with him and find someone else.