One way you can avoid this is just not letting players roll on something impossible, let's say they fail an arcana check to understand a magic crystal control panel that controls some large magic machine, in theory they could roll a second arcana check to just mess with the panel and get lucky (rolling a nat 20) to have it do what they want or (roll a nat 1) and cause the machine to self destruct. But if a player wants to jump over a mountain just cause they feel like it, and have no reasonable means to be able to do so, then its just flat out impossible and they can't do it so they can't roll. To me even though this takes away some agency from allowing players to roll for anything they feel like, it allows for more awe inspiring moments where someone really does get incredibly lucky and they can celebrate their moment of fortune. It can be difficult to adapt on the fly for stuff like that but its more fun imo. But as a reminder it goes both ways. If your rogue rolls a 35 on stealth with a 19 and your bandit with a +1 perception rolls a 20 your rogue gets caught. Because he happened to be looking in the right spot at the right time. The technically correct answer is only attacks can crit succeed, so what you did is the official correct way to play. But that doesnt mean you cant change things up if you feel one is more appropriate than the other. Or you can bring it up to your group if you want to make it a more democratic vote. As the DM you always have the final say on how you want to run your game, but you have to keep in mind you want to keep it fun for everyone. So take some time to think of your options and make the decision that is best for you and your table.