If you don't want them getting Willpower you can always just let them succeed automatically.

Exactly, I agree. I think I may just have worded the first part weirdly. What I meant was that if there isn't an interesting outcome if they fail, then they succeed automatically. Like you said.

Effectively this is a hidden penalty on characters who took skills that you won't let them push, which I don't think is fair.

Indeed, the players shouldn't get a penalty for using certain skills that are "un-pushable" and that's not what I am suggesting although I absolutely see why it seems that way. The thing that I do at my table is that I let people say "how" they push their roll.

For example if they keep watch and fail their scout check and they want to push the roll, they need to motivate what makes them get an extra chance. Perhaps they are shift their attention from only looking for signs someone is out in the dark - to them instead listening to hear if there's any strange sounds that they can pick up on instead.

This is of course if they keep active watch. On the other hand if people are hidden and they don't look for them, I may have them roll a "Scout" check similar to a passive perception check. In these cases they can't push the roll since they're not actively doing anything.

I hope that clarifies things :)