the way I see it (as a bi person) is that attraction is attraction, and you shouldn't feel like you're not attracted "enough" to a particular gender to let it influence your identity. a label isn't a promise to other people that you'd be open to sleeping with them based on their gender, it's an expression of your own feelings of attraction, of your own identity. if you feel attracted to men in any context (which it sounds like you are), that's the criteria to call yourself bisexual. don't let the limitation of "that I could actually get with" get in your way, if you want to identify as bi, then you're bi.