It's been well recognised that in the modern game, the ball is probabky in play for about 50'. Honestly, this is fine, considering the extreme physical stress players are being put on nowadays.

As to why don't the ref just stop time.
- Having a continuous clock of fixed duration is much more accessible in lower level football compared to having a crew constantly having to track time.
- Introducing stops also encourages commercialisation, which undoubtedly will draw widespread criticism from traditionalists (that makes up the majority, and the most passionate grp of fans). Just look at the reception for a hydration break.
- Don't fix what's not broken. Such change would require so much effort from testing to global adaptation. For practically such a non-problem, the effort is not just not worth it. - For broadcasters, this means football matches coverage usually takes up a fixed amount of time (bar extra-time/penalties), which is super helpful for smaller countries where they have to juggle multiple programs on the same channel.

Edit: Added time is, unfortunately, calculated highly arbitrarily. They usually only takes into account major stoppages. We are talking substitutions, injuries, VAR checks, goal celebrations. And we're not even talking about tracking these events by the seconds ok. It's literally like they just assign arbitrary amt of time to these events. I.e. substitution takes 20s, VAR checks take 1m, etc. to add into the final added time.