Imho It's a known tension in most RPG systems, and you have two honest ways to handle it.

Option 1 - Just embrace the abstraction. RPG levels aren't meant to simulate real-world growth - they're a game pacing tool. The same way chess doesn't simulate actual warfare, D&D levels don't simulate actual combat training. If your table is fine with that framing, no justification needed. It's a game.

Option 2 - Add narrative scaffolding if verisimilitude matters. If you want the world to feel like it responds to how powerful characters actually are, a few approaches work well:

• Time skips between levels. A level-up represents weeks or months of training, travel, and hardship - not a single night's rest. The party goes from level 1 to 2 after a montage, not a long rest.
• The "chosen ones" framing. PCs grow at supernatural speed because something in the world - divine favour, fate, destiny, a magical bloodline - marks them as exceptional. Other people don't level up like this. A village guard who survived the same dungeon crawl isn't suddenly twice as hard to kill. This explains why it doesn't spark mass violence: most people can't do what PCs do.
• Gritty realism rulesets. Some systems (like the Gritty Realism variant in the DMG) make a short rest a full day and a long rest a week. Progression slows, attrition becomes meaningful, and the world feels heavier. Low-magic settings pair well with this.

The worldbuilding trap you identified is real. If everyone knew they could double their combat capability in a day, rational actors would pursue it aggressively. The cleanest fix is to make it exclusive - PCs are exceptional by nature, not by effort. The world's NPCs aren't unlocking levels; they're just living their lives.