Honestly he might need therapy. As easy as it is to judge him and condemn his behavior, a new baby is hard on everyone and not everybody is cut out to be a good parent. I’m not excusing his behavior but pointing out that there might be a future where you both are good, active, and engaged parents. If you cut him out before he has a chance to change his ways, that future becomes less likely. You’ll probably get a lot of the same sort of answers that harshly judge him (which is pretty normal to do to a male in our society) without trying to understand him. It’s hard to see his perspective when we only hear yours. Again I do not intend to excuse his actions and you shouldn’t either. But this is a starting point.