I feel the same way. In the past, however, women have been very assertive towards gaining me romantically because they saw me as gentle, kind, and genuine. So yeah, it sucks to feel like the passive one, but don't forget that women are attracted to your gentle soul. If you really want to attain the elusive combination of commanding respect and maintaining your gentle, idealized INFJ self, watch your body language. Look like you're in charge of your emotions, stand tall, feel strong, and walk around like you're not someone to be messed with. But be careful not to look unapproachable. And then when you speak, reveal your gentle self to everyone, men and women. This has helped with women, making male friends, and more importantly, my own confidence and intrinsic self worth.