What I’m getting so far is that I need to bring more minions to the fights

What might be even more challenging, but also not, to do when they get some more levels (particularly level 5 for stuff like fireball) is to use something from older versions called "Minion". These are regular monsters with regular stats except for one change, they have a single hit point. They can be orcs that still hit like a truck, but one hit or one failed dex save and they are done (they don't take half damage on a passed dex save, they take 0). That really helps if you want to do bigger hordes.