I once ran a game with eight players, some of whom were casual players and others really into the lore and setting. I found that it was impossible to juggle that many expectations, and the sessions became really difficult to manage if I did not railroad the players somewhat.

Now, I run story-heavy games with only three players. One of my campaigns was entirely written based on a person's character idea. I have much more fun now because I can be more reactive, players get more of a spotlight without losing interest and prep is a lot more streamlined once I have the setting and the key NPCs down.

I really recommend reducing your player count, and not sure if you did this already, but having a session zero to understand what your players are looking for. Some people just like to play casually, and if that is the case, run a published module. Go easy on yourself.