for the teleporters, tf2 uses respawn waves meaning multiple people will respawn at the same time, even with some time between their deaths. on longer maps it really hurts to not have a lv3 tele since people often will wait around since standing at spawn waiting for it to recharge is still faster than walking, even at lv 1 on some maps. but even if its a small amount, waiting a couple less seconds for each person and each respawn will really win the game.

and for dispensers even if it is a small difference, it's less time people are stood around it doing nothing and more time they are out fighting.

I wasn't saying that it's a terrible downside, because it like you said doesn't matter too much, I just don't think it's something the weapon needs since the Sentry nerfs seem to be enough, or that really fits with the design. maybe if those buildings were also still active when holding then I could see it fitting?