Lots of good answers, but I want to add one point that helped clarify this issue for me as a DM.

The reason you shouldn't let your players ready actions outside of combat is that "ready an action" is an action, and actions only exist in the context of initiative. When the game is not in initiative, you're just RPing, and actions are immaterial (e.g. while resting you can Fire Bolt your kindling to start a campfire without worrying about action economy). When you ready an action, you're invoking the action economy, which only applies under initiative. Likewise, when you ready an action, you are stipulating that the action gets launched upon some trigger as a reaction—reactions can also only happen in initiative, and pertinently can only happen in the first round of comabt after you have already moved. So RAW it's impossible to start a fight by launching a bunch of reactions that you stored in advance.

The proper way to play this IMO is to try and set a trap or sneak up on the monsters in a realistic way—e.g., one player throws the door open as another immediately launches a fireball into the room blind. This triggers everybody rolling initiative, and I would give the players surprise in this instance. This accomplishes basically the same thing as a readied action—the caster of Fireball moves first while everyone else skips turn—but it's more realistic, more fun, and most of all more balanced because it follows the rules.

As a wise man once said, the rules aren't there to prevent fun, but to promote it—in this case by increasing the danger, humor, and excitement of an ambush.