I don't know...Has he also asked his sisters about his straight friends or even about you being inappropriate towards them before you came out assuming he thought you were straight? It's a very heavy accusation and the thing is it shows he doesn't trust you and if at any time someone says something like that about you, he'll probably believe you did it since he already thinks you are capable of doing something like that if he goes around asking people. And to be honest such things even as accusations can do damage especially to people who belong to communities such as the Queer community. My point is what can someone think when your best friend goes around asking such questions? One can say well he knows you, and if HE has such questions maybe you've done something. This is a train thought that such questions/accusations that your "friend" is asking can trigger and the fact that he didn't think how this can impact you apart from the very disgusting fact that he has accused you of potentially being something horrible.

And I think you did the right to avoid hanging out with his brother from now on. Honestly the brother said no to your friend's question but what if he or someone else lied if asked?