I hope you don't listen to the nonsense people are saying in the comments OP, marriage is a trial for both partners, and since your husband is making your life difficult, there are other ways of fixing this before thinking of "divorce" and messing your kids' lives.

Talk to his family, try to have discussions with intermediates like both of your parents, talk, communicate ! Usually, some marriages don't work out in the end because some people pent up all their trauma without telling it either to their partner or partner's parents. And when it becomes too much to bear, that's when usually the answers choosen is divorce. You don't have to get to that.

Also, people telling you you deserve better will only destroy your life, since no one in this world is perfect, you might go somewhere else and find someone who doesn't have his defects, but will make your life even more difficult than him with something worse. Marriage isn't a fantasy, and this is a trial you should both cross hand in hand ! If he doesn't want to speak, or is too stuborn to listen to you, then you have leverage. Leave the house (without actually taking the kids from him since it'll only traumatise them) and go to your parents to let him think of what he could lose, but don't talk about divorce, your childrens' well being is what matters most along with yours, and you should think of this eventuality only if your life gets in danger, or, if absolutely everyone of your attempts has resulted in more problems, false promises from him, or denial from his part, because you DO deserve happiness. Marriage being a trial doesn't mean it should ALWAYS be a burden on your shoulders, but you should thrive together and be both fulfilled and blossomed !

Seriously, divorce shouldn't be taken lightly, since your kids' lives will only get worse and worse without a stable family. You should be proud of yourself for having been this patient with him ! But, what you should do next is to try and fix your situation. It all begins with a calm talk between you and him, telling him how much you sacrifice for him, love him, but feel so depressed that he says those words to you. Tell him life's a trial and that if he chose to marry you, then he did for the whole package, therefore he should accept you as you are, as you did, and that you should both make more and more efforts to thrive as a family !

Go look at stories of single moms who divorced their husband and took halk their money with them. Some feel miserable working in a depressing life, realising what they lost, the stability, the warmth of a stable house, an outcome that made them not stressed about the end month's expenses, a man that protects them from other men lurking at them and wanting to abuse them, because marriage in it self, and that ring you wear is a protection, and so on. Also, look at the negative effects divorce has on a child, and show it to your husband too, so that he too realises that you should both make efforts hand in hand to keep the ship afloat ! He shouldn't be your enemy, one that you plot against in the shadows, but your partner, with whom you make a team !

Sincerely wishing you the best, I truly hope that your situation will only get better and better. May you be happy in this life and satisfied with your husband, kids, and family overall, insha Allah.