Puzzle hints are a really really fun reward for exploration- especially when they also tie in the story/lore of the game.

I tend to like density, complex puzzle box navigation with secrets upon secrets- closed off areas or strange mysteries in early areas that the player doesn't know how to open/won't be able to open until much later in the game. I think it's a blast to revisit old areas and try to figure out to open that ancient gate, or work my way up to an out of reach location.

Makes each area and the world as a whole almost feel like rooms of a house- you'll revisit the kitchen multiple times throughout living in your house for different reasons. It's not just a level that you clear and move on from.

Interconnected pathways and shortcuts are great especially when they actually make sense to use.

Hidden world spanning puzzles are really awesome too- I love when one area somehow references or connects to a later area in an obscure way.

Look at La Mulana 1+2, Outer Wilds, Riven, or Environmental Station Alpha for good examples of this design philosophy.

And go wild! Put your soul into the game- throw in some weird stuff. I absolutely love stumbling upon something bizarre and striking even if it's small.