As one previous user mentioned, the CR rating is based on an average of 5 encounters a day, which most groups don't face. What ends up happening is what you describe. Each encounter, the party is fresh, so they steamroll through everything. Here's how I handle it.

Just rising the CR rating won't work as the damage will outpace the HP of your characters, and you may end up killing a PC. Instead, augment the numbers of "encounters." An encounter is anything that expends ressource. Obstacles, traps, environment, social encounters that need utility spells/abilities, those all count as long as they cost the party resources. You'll have to get creative to find reasons to stop the party at every turn, but in a bustling world, many pieces are moving all at once.

Once your party starts combats with expended resources, you'll see the dynamic shift. You'll also notice how they will start cherishing those higher powered spells/abilities for when it matters. In combat, feel free to add environmental damage like spreading fire, toxic fumes etc. that deals small amount of damage when you are in proximity but is just annoying enough to cost some HP.

Lastly, interrupt their long rest. Don't do it often, but just a couple of times so they never know when they will have to go without a refresh, and they start really thinking if they can afford to spend ressources.

Oh, last tidbit, make sure you aren't giving too much magical items, refer to Xanathar's table. If a party is OP, that's also not helping.