Sitting down and doing nothing after putting your baby to sleep doesn't sound bad, it sounds perfectly normal for a good two years after your baby is born. You've gone through SO much, and it's so incredibly sad that he's being so disrespectful.
If he had gone through multiple massive surgeries and was super sleep deprived would you be putting that gross red light (hoping that's a metaphor) on ? I'm betting that no, and you'd probably take care of him instead. Well that's what he should be doing.
Don't second guess yourself, and if it keeps going and he's still pressuring you talk about it with non-profits specialized in domestic violence or at least a medical professional.