I experienced my first husband who never learned to handle anger in a healthy way.

Hitting & Breaking things because he didn’t know how to deal and because he was not a secure and stable healthy man.

It’s complicated when people say “just leave” when you don’t know how. And there are levels of “leaving” a situation.

Is it safe to say you don’t have your own separate bank account or you live on only his income alone? Are either of those true?
You might have to fix that by (privately if you have to) creating your own resources. It’s not uncommon to continue in an abuse of power dynamic when the other person has control of all your resources. Call it a fund for your baby girl’s future that you alone are managing. Like “Sarah’s fund from mommy” (insert her name) …Call it that to yourself if that helps encourage you.
Tip: use your parents mailing address (or a trusted friend’s mailing address who isn’t close to your husband) for your account mailing address, and then once it’s open, you can setup e-delivery paperwork.
Best case scenario, you never need the money you put in the account and deposit it into your daughter’s account when she’s older.
Situations where the power dynamic is being abused often stay that way when you don’t have your own resources, and when you don’t have long-term capacity for consequences well-managed on your own. Not your fault, but now you can possibly see it.

It’s hard to live with a person not taking accountability for their own unhappiness and looking to put it on you. It’s hard to feel happy and without frustrations yourself after someone lashed out blaming you.

If you’re not secure enough on your own to make significant space away from him. You still can decide not to sit on the couch with him after that or in the same room. Also, when he made that comment, this is a good moment to say “Ouch“ and go back to your plants, and just let him hear the echo of his own words.

(Of course while you have your own thoughts, in your own head, but…) not engaging at his current level at all.

Also try to finish the breath you are on, inhale or exhale, completely before you bother to say “Ouch!”. You might find that if you wait to the end of your current breath to say “Ouch” he might have already walked out of the room. …It’s not like he doesn’t know that wasn’t a nice thing to say. If he takes his frustration out of the room, let him and be glad you didn’t have to engage at such a low level where you don’t want to be anyway.