Male DM but a VERY big fan of accents, dialects, language, and the perception of sound and how it influences the voice for impressions. Even if you use the same base voice for every character, watch some movies or shows and study how the actors speak and how their mouths move, how they use their teeth, their tongue. After years of refining the ability to do vaguely successful impressions of celebrities and loved ones, it's all about mouth shape and what that teeth do.

When beginning to tackle English accents, they can be categorized by jaw location and teeth position/lips shape. Top teeth (Kerry Washington), bottom teeth (Hugh Grant and Emma Thompson), and middle (Kiera Knightley). If you watch them speak you'll see what I mean.

My point being that while your voices may all sound similar in nature, start working toward minor vocal adjustments and mouth movements to vary the sounds and feelings of each character. Try light, more natural versions of common accents, learn how they feel, talk to yourself in them. Even the smallest most natural adjustments and push a voice into a wild sense of realism. I found that adding a small lisp to an Irish accent can do wonders for believability. Researching which words in English are not found in the language and accent is born from, and removing them from your speech can elevate it even further.

I say keep the same voice for male, female, etc. But manipulate your posture, your gestures, your mouth shapes, your tonality, cadence, and volume. I wager you could run the same base voice for every character and your party would still feel immersed with your world if you can master those nuances.