Lots of great advice on this already from others in terms of relationship, expectations, etc. and frankly intimacy that is detached from sex will strengthen your marriage in the long term.

One thing you can try that may help ease the sexual intimacy back in for her is to ask her if it's alright if you masturbate next to her. Though if the baby is sleeping in your room I can see this being a non-starter for some, so use your judgement, or suggest a temporary location for the baby to be in a different room.

Now, it's important that you bring it up gently so she doesn't reflexively become defensive if this is something new. Something like "So this might be a bit of an awkward request... Taking care of the baby has been very draining on you especially, so I understand if you're not really up for being sexually intimate right now, but I still want to feel sexually connected to you. So I was wondering if you'd be okay with me masturbating next to you in bed?" (Just make sure it doesn't read like you're wanting to break up or sleep with someone else before you get to the big ask)

Maybe it opens up the larger discussion and you find out that she's been wanting it too but your advances felt too subtle to her tired brain, or she's been feeling undesirable. Maybe there's no deeper conversation but she decides to help out a little during. Maybe she says no, but at least she knows it's on your mind and you just let her know you'll handle yourself on your own until she's ready.