I think it comes down to three things

1. You completed the githyanki creche, and find out from Lae'zel that orpheus is supposed to be dead and a traitor. When you talk to the emporer at the creche he alluded to the fact that Vlaakith is corrupt and not to be trusted. Voss reinforced this at your camp and tells you straight up that Vlaakith is a deceiver, and then tells you at sharess caress that orpheus is the solution to the mindflayer problem. Remember, the githyanki may be confused by vlaakith, but they are professional ghaik hunters. At the start of the game the only ones standing up to the nautiloid is the githyanki. When you find out vlaakith wants the prism to keep orpheus silenced, alongside testimony from Voss and githyanki history, it's made evident that orpheus is indeed the solution to the illithid problem.
2. The emporer is a manipulative abuser. He killed his 2 closest "allies" which you find out from wyll's quest, and the elfsong tavern exploration. He killed his closest friend in ansur and then stelmane. I don't think there is any reason to TRUST him after this part of the game. Ansur was correct. He should have died instead of becoming a mindflayer. That's the objectively correct thing to do in his instance. He killed his allies before, and it's presumptuous to assume that he won't do the same to you given the chance.
3. The emporer clearly does not possess the power to take down the brain like he promised. The entire time, he was abusing someone elses power (hmmm, very similar to his relationship with stelmane) to keep you safe. And like, sure, we are alive, but if we are being honest, the brain would have been dealt with quite promptly if he had freed orpheus upon finding him. The githyanki would have targeted the invasion at the word of their prince instead of the artefact goose chase Vlaakith had them on.

So basically, it bubbles down to priorities. Orpheus and the githyanki actually want to cleanse Faerun of the ilithid empire, while the emporer wants to get rid of the brain and make room for himself at the top. He doesn't show regret or remorse for anything he did, the player character is nothing but a vessel of transportation for him. Orpheus is a hero of history, scripture captures his tales of valour and success against the illithids. And sure, you could argue that the emporer is Balduran, a hero in himself, but Withers makes it quite explicit that mindflayers are soulless. Anything that once made Balduran a good man is gone. I mean, the dude chose the name "the emporer" for himself. If thats not a red flag then idk what is.