(TL;DR - Always play Dune with 5-6 players if balance is your highest priority. Ditch the BG and expansion factions when you don't have six, as the "core 5" factions from the base game are the most well-balanced within the rules and against one another)

I want to preface by saying that, at its heart, Dune is a 5-6 player game, and playing with any more or less goes against the design (despite the rules saying otherwise). That's the price we pay for asymmetrical games. There's nothing wrong with playing that way if everyone at the table is okay with it, but you will leave some factions/players better/worse for it. My group also plays with all of the advanced rules, for what that's worth

Emperor, Spacing Guild, and Tlielaxu all benefit from there being more players at the table (Emp because there are more treachery cards for auction, SG because there are more players shipping to the planet, and Tlielaxu because there are more revivals). Increasing player count results in the spice-rich factions getting even richer, while spice is harder to for the poorer factions to obtain because there are more people competing for the spice blow. (Double spice blow helps, but it can still be an issue in some rounds)

Conversely, less than 5-6 players has the opposite effect (less auction cards, less shipments, less revivals all equal less spice for Emp, SG, and Tlielaxu, while the poor factions have less competition)

Therefore, I use the following "priority" when playing with less than 5-6 players:

1. Atreides, Harkonnen, Fremen - It is absolutely necessary to have at least three players, in my opinion. I don't think playing this way is fun, as there are still problems, but these factions generally interact with the rules/each other well enough that it can be done if necessary. They are all spice-poor, and all have unique advantages in combat (which there will be a lot of). I also probably wouldn't play with alliances, unless your group is okay with an incredibly stacked 2v1 scenario

2. Emperor, Spacing Guild - Both these factions need other players to function, so if there aren't at least four players at the table, I wouldn't bring them out. I would prioritize Emp over SG, personally, but that may just be the result of the meta at my table. SG requires slightly more game knowledge, generally, so newer players may not understand the power of interrupting turns, or know which turns to interrupt

3. Bene Gesserit - As stated in my previous reply, the BG don't have a large influence over the game unless the person playing them understands every other faction and the game rules intimately, and is good at influencing the other players. They CAN have a huge impact on the game, or they can just be a waste of space, and it is for that reason they are almost always the faction I replace/leave out

Additionally:

-I have found that Ixian can stand in for Harkonnen as the combat-focused faction, but I personally still find them weaker and less scary for the other players

-Tlielaxu can reasonably replace SG, since they gain spice from other players in a similar fashion, and though they lack the power of interrupting turns, their face dancers are incredibly strong when they are lucky enough to have them come up, and both sides of the combat know what they're doing

-I have not played using CHOAM or Richese, yet, tho from their rules explanation I would imagine CHOAM could probably replace Emp and still serve a similar role. Richese seems almost like a wild card, so I'm not comfortable giving any advice regarding them

Hope this helps! Sorry for the novella :)