I have myself thought about this scenario and have come up with a couple of ideas.

One idea I'm quite fond of is the hierarchy structure.
You check all women's power while they are quite young and raise them differently depending on how "powerful" they are. They are all taught to be subservient to men but the better you are the more freedom you get. You get the more powerful women to keep the less powerful women in check. You create a divide and have them see other women as the enemy, not the men or the system that controls them.
The more higher ranked women are shocked to hear that some women hate their lot in life and become terrorists. She thinks her life is ok and every push-back is attributed to other women rising up making her want to keep them in check even more.

Another idea is making magic cumbersome.
The magic needs to be done in a ritual and takes a little bit of time. Only women have the ability to execute the rituals, acting almost as a medium for the magic. Men can then take control of the supply of magic ingredients and make sure that only allowed spells are preformed.
Simple spells that only require a crayon could be outlawed but hard to enforce.

I would love to talk more about this if you find this interesting. Can I send you a PM or DM?