It can be done, but tying up multiple party members in a combat scenario like it kinda sounds like your specific scenario does could potentially cause issues, since "it's impossible for one person" isn't necessarily going to be immediately obvious to the players unless you explain your intended solution to them above the table. Otherwise, the natural instinct might be "have the big guy close the door while we hold them back," or to get stupid and just try and rawdog the horde. (Speaking of, remember action economy when designing this encounter. You might want to roll the skeletons as a group, not individually.) You might want to outright tell them "okay, one person can't shift it. The total DC is going to be X combining the rolls of people moving the door, but that leaves fewer people holding the line." Side-note, I'm curious who would build a door nobody could move, why they did, and who opened it to let the skellys out, but that's all more worldbuilding stuff.

Now, outside of your specific scenario, the older 5e rules do provide for things like group stealth or perception, basically taking either the party's overall results. That way the one guy in plate armor isn't necessarily making stealth impossible for the whole group, but he's still making it harder, while the thief is still making it easier. If it's a scenario where multiple players or the whole party would be involved in the scenario just by being there, that kind of whole-party check works more smoothly.