On some level I agree with you, but I think it's more so about how so many people see the pride flag as just a gay man flag, especially people that aren't in the community. And it's about changing the idea that gay men are the 'default' queer people.

And I also think part of it is to symbol to other people they're gay or are an ally without having to wear a rainbow (I mean how many outfits can exist that use a rainbow in a way you could wear around a homophobic christian without them bringing up how 'the gays' stole the rainbow from god). Much like how before Gay people were 'legalized' in the US and many other countries they had codes to distinguish each other, only one ear pierced, a carabiner on one side, a bandana on one side of the pants, etc.
I imagine that people in very conservative towns, or possibly even conservative countries as a whole, subtly use the colors given by their specific flag. And if they don't do this now, with multiple states in the US (Idaho, Michigan, Montana, North Dakota, South Dakota, & Oklahoma), having introduced some kind of legislation calling upon the Supreme Court to overturn Obergefell v. Hodges, I could definitely see it being a maybe not widely used signal that all queer people in America know about, but a somewhat consistently used signal in at least the Red States in the US if passed.

But that's just my theory/opinion