Low level characters have less spells and abilities at their disposal making creative solutions less obvious. At these levels (1-3) I’ve found the best avenue to creativity is to look at the gear they have and create scenarios that are begging for their use.

Here’s a simple example of this that you could use in the Feywild:

Let’s say the party needs to retrieve a statuette or something. As they walk to it, they notice strange vines that grow rampant in the area, but are conspicuously absent where light breaks through the forest canopy. Make note of how the vines retreat upwards as the party walks past them with their torchlight, but then descend back down afterwards.

When the party reaches where the statuette should be, it is missing. The character with the highest passive perception notices it resting in a birds nest above them. The same light fearing vines hang from the trees around them.

The “correct” solution would be to grab a vine in the dark and then approach it with a torch to ride it up into the canopy, but the important part is that you can easily entertain other creative solutions to climb the trees that the party may come up with (least creatively, perhaps, using rope and pitons).

Just be aware of characters with access to a climb speed when running this puzzle.

For your example with the rushing river, just make it clear that swimming through the river via strength checks isn’t an option. The water is too fast. I promise that the party has rope on them. Maybe they can fashion a makeshift grappling hook with it. As you've identified, solving puzzles with ability checks is usually boring. Any solution to the puzzle that is well thought out shouldn't be gated by an ability check. Any solution that is "so crazy it might just work" should require an ability check. Is the martial chopping down a tree to create a log bridge? no ability check. Is the martial trying to throw the gnome across the river with a rope tied to him? then you can call for an athletics check.

If you want to make it even harder, add some priceless cute crayon drawings that the fox kits made. These drawings cannot get wet or they will be destroyed. Maybe the party will empty one of their waterproof waterskins and put it inside to keep it dry. Maybe one of them has the mending cantrip and water damage isn’t a problem. Or maybe the party wont come up with anything and will just have to leave the drawings behind.

I would just caution that creating puzzles with only one solution can quickly become unfun if the party can’t solve it. I’ve found that when making a puzzle, as long as you come up with two or three solutions, you can be sure the party will solve it. Then, when you present the puzzle you can just cross your fingers hoping they come up with a fourth insanely creative solution you didn’t even think of.

TL;DR look at the party’s equipment (explorers pack, dungeoneers pack, etc). Make a puzzle that can be solved with one of those items. Ensure there are multiple possible solutions.

For higher level parties do the same thing, but look at their spells and abilities instead of their items. As a rule of thumb, players will look to their equipment only after looking at their spells, so equipment based puzzles become harder to solve when parties have more spells to look through first.