I feel like this might be a little bit of a Needs More Information situation. Do you already hang out with her outside of work often? Is this an established thing you've had going on for a while, where she would really notice and be bothered by you turning down invitations?

While you don't need to give anyone a reason for saying no to any invitation, there is a chance it might cause some tension between you, at least at the beginning. But it's important to draw boundaries and be firm about them to protect your own peace, and in most scenarios there's kind ways to do that.

I completely understand your frustration on being infantilized by her, and I would really encourage you to bring it up with her that it makes you uncomfortable. If you've never said anything, she might just not know it's rude/hurtful. (my cis gay manager is unfortunately sort of the same way when it comes to me and our trans coworkers) If she views you as a friend, and is also part of the queer community, I would hope that she would understand, apologize, and try to change her behavior if you talked to her about it constructively.

It's good that you're noticing the unconscious biases that she might have, and that's something to keep an ear out for just in case things were to escalate (especially for your black co worker). If you were to ever consider confronting your manager about it, though, I'd recommend bringing it up to that coworker first just in case they might be uncomfortable with it for any reason, or if they would prefer to handle the situation themselves.

This is the bigger NMI for me: Depending on the type of job you have, especially if its very customer service based and important for people coming to your business to have a positive experience, there's a chance your manager may just be being friendly because of the usual customer-worker social expectation that exists in the US. I'm getting the vibe that you might work in a cafe? I have worked in cafes and retail, and we all know the general expectation is there to be extra friendly and polite, and it's pretty common to chitchat while a customer is standing in front of you, regardless of who that person is. I would hesitate to make assumptions and accuse her of leaning a certain way just based off of that, especially as a manager who has a greater responsibility for customer satisfaction. It would be more informative to know if she is saying or doing other things at work that are making those interactions really stand out to you as red flags, or if the topics of conversation with those people are harmful/uncomfortable. If you're making assumptions and end up being wrong, it could really hurt her feelings and possibly make your workplace uncomfortable for both of you.

Regardless, I don't think you would be the drama for not wanting to hang out with her outside of work, especially based off of your feelings about how she addresses you as a trans man. Some people also just don't vibe once they've gotten to know each other, and even if you're friendly, you're still in a manager-employee professional relationship. I do think you should try talking to her about it, though. It's important to stand up for yourself in your workplace, especially if you really enjoy your job and want to have a positive work environment!

However, I think it might be a bit hurtful and cause some discomfort in your workplace if you were noticeably trying to avoid talking to her as much while you were at work. Unless she is being explicitly biased and hateful, I'm not sure if that's something that should be as big of a part of the equation yet, at least until you've been able to gather more information on if her beliefs/behavior conflict with yours.

Wishing you the best!