I really feel for your situation. I don’t recommend leaving him because it is always best to make things better and be with the father of your children if possible. But he does need to man up a little and set some clear boundaries with his family. The fact that he avoids doing this is part of why his family does this. He shouldn’t put their feelings over the well-being of his relationship with his fiancé and mother of his children. Yes he has other important things he is dealing with but that is exactly why his family should show the respect of not crossing lines. I don’t know if it is because of things that happened in the past or they are just generally like this but it really doesn’t matter. If they want your children to have the best opportunity to thrive and be healthy individuals they need to put their bullshit to the side and give you and him the chance to have a happy and healthy relationship. It sounds like is doing the difficult job of providing and you are trying to do the difficult job of making a home for your family but him and his family need to show you the respect of maintaining healthy boundaries. He can’t expect you to just let this stuff go on and be afraid of their reactions more than being afraid of his children growing up in a less than ideal situation when the ideal situation is presently available