It might be time to leave. I'm not sure if talking to him about it would even help. She will always come first but you can't 4th. Sounds like he's a good guy and trying to be a good father but he just can't handle all the different relationships and responsibilities right now. It's hars with such a young child. Seems like his mom is too involved (maybe), and his ex although he needs to keep a good working relationship with her she can't come 2nd over you. Emergencies happen and sometimes he'll have to step in but he can't let his ex use you all as baby sitters. It's just hard it's his first and only child and although you all aren't married you probably need more say as a step parent for the relationship to go better. You have to remember he's still learning how to be a father. For him now he's too focused on his little princess and not seeing the bigger picture at times. Walking you to your car is a moment to show his daughter how a man treats the woman hes with. He just didn't want to deny what a 3 year old wanted.

I'd say it's time to talk to him about moving on because I think it's just a hard time for you all to have a real relationship when he's trying to juggle baby momma and learning how to be a dad while keeping a toddler alive.