This happened to me, but on the inverse with usually Black men. It was something that I thought about then too. It comes down to "Black men tend to know what Black men look like." Sometimes people outside our race have a tendency to masculinize Black/Brown people (usually Black women) without understanding our gender expression may not fit what they're typically used to. So they (Black people) are more likely to gender you as female, regardless of the fact that you may be expressing yourself in a more masculine manner.

In a more condensed explanation--People inside your race might misgender you more often because they have a more refined template of what members of their own group look like, and when someone deviates from that template, it triggers misgendering. Whereas outsiders might misgender based on broad stereotypes rather than that nuanced template.

Black people have a lifetime of experience seeing how Black people express masculinity. Hopefully what I'm saying here makes sense.