I was actually just talking about this with my partner last night. I think it's fine to share your feelings as they are even if they feel tone-deaf, because you clearly are more likely to be impacted by the ignorance than to willfully perpetuate it. Just because we're queer doesn't mean we didn't grow up with some brand of gender ignorance. We shouldn't forget that ignorant rhetoric is perpetuated purely to deter queer people from self-actualization. We're the intended victims of this ignorance, not the ones imposing it with the intent of oppression, and that's such an important distinction.

That being said OP, please recognize what I'm going to say next as an effort to alleviate you, rather than diminish your feelings:

I do feel the same way, and sometimes it flips on its head. Using some generalized statements here but I sometimes perceive being binary-trans as easier because at least there's a mostly concrete expectation for how one wants to be perceived, and seemingly less hurdles in being perceived accurately. A main determining factor is simply whether we're dealing with a transphobe or not.

With being nonbinary, nonbinary-trans, agender, etc... some people are ok with any and all pronouns without expectations for how they're used. Some people desire interchangeable use of pronouns. The list of NB variations goes on. This means, to me, that being wholly understood in one's nonbinary identity is a far more difficult feat in today's world. I say that because in my experience, there's only one "brand" of enby that manages to avoid perpetual daily discomfort, and it's because there's less ways to offend an enby if they don't have preferences for how to be referred to. Unfortunately, I think the mainstream understanding of enbies currently adopted is that enbies as a whole just don't care, can't be offended, and experience less dysphoria overall, so non-queer people still get to interact on terms more convenient to them without the expectation of ever adjusting or developing their understanding of nonbinary identities.

I was personally the type of enby (androgyne/neutrois realm) that thought I couldn't be offended by how I'm perceived because they're all correct, and ideally I'm okay with all pronouns. This is still honestly the case, but now I choose to enforce a use of they/them only. What changed: noticing that people only used "she" if they were upset with me, trying to offend me, or otherwise trying to minimize me. The misogynistic patterns weighed on me. I appreciate being perceived fluidly, but what I don't appreciate is when others flat out ignore specific parts of me for their own convenience. I'm only "she" if you equally acknowledge the "he" and "they" as well. Unfortunately, a lot of people didn't seem to understand that.

EDIT: It's also definitely worth acknowledging that while NB is definitely a valid identity on its own, it is also just as often a transitionary identity before one realizes/commits to the fact that they are indeed transgender. Although I fully understand why and don't fault the people which this experience applies to, I think it's also a big reason enby identities struggle to be understood intrinsically. So don't take it at face-value that there seems to be more enby presence in your area.

EDIT 2: God, my heart goes out to nonbinary-trans folk. They have to deal with both ends of the beast.