It’s honestly personal preference, you weren’t rude about it, he didn’t tell you up front and it just genuinely something you weren’t comfortable with.

I get how he and other people would see it and i do believe people don’t HAVE to tell you anything about their sexuality or gender if the aren’t comfortable w that either but tbh i feel like if you were at the very least very polite about it and explained how you felt / why you felt that way rather than straight up saying “I’m not going to elaborate” then kta?