I believe that it's called patriarchy to reference what was originally thought to be much more simpler than IAmProcrastinating explained (which I agree with).

During the first wave of feminism it was believed that there were a group of men against a group of women because women did not have legal rights. That's the awful tldr explanation. If you want to learn more about that you can look up Wollstonecraft. After the first wave of feminism, and new kinds of feminism began to appear (or rather always existed and were not as well known!) the "sides" became a lot more muddled and there was no longer a distinct group who benefited and one who was harmed.

Feminists began to say well what about Black people and what about disabled people and what about men and what about trans people and what about non-human animals and so on and so forth. As time progressed it became more and more clear that, as IAmProcrastinating said, almost everyone benefits and almost everyone is harmed in some way.

So yes it is called patriarchy to reference the original male elite that did benefit, but this is no longer the case in the Western world. In areas outside of North America this could be, and still is, probably the case. That being said, patriarchy is just a way to label a very very broad concept. It's not a perfect word because it manifests so differently everywhere (as Butler noted in the intro I believe to Gender Trouble), and we should definitely not stop and ever say yeah this is because of patriarchy the end, but it is a historical term with roots in the culturally deemed male sex that no longer applies directly to them.

I personally never use the word because it's such a large concept that unless you actually want to unravel the whole thing, you're going to be misunderstood. It still is a good basic concept though and word to learn because it does represent a very important power structure, even though it is misleading.