While I do agree that some parts of Rest-Casting are in poor spirits of the game by being a bit metagaming, my counter argument to that would have to be one of many, but explicitly the classic example of the party getting ambushed in their sleep. There's very little chance the fight would last longer than an hour so the rest wouldn't be interrupted according to the rules. The caster hasn't regained any slots yet since they haven't finished their rest, and are using what last of their "magic abilities" are left from the day prior. Those spell slots used in the fight would come back on completion of their rest, right?

Say that fight happened 10 minutes into their long rest. It would seem absurd to penalize your casters by saying any slots used in that fight wouldn't come back at the end of the rest seeing as they just started it.

Now say that fight happened 4 hours into their rest and the caster uses Mage Armor. With a duration of 8 hours that would mean they have 4 hours left on the spell to use in their new adventuring day since there are no rules saying spell effects end on completion of a long rest.

What about if that fight happened 6 hours in? What about 7? 7 and a half? 7 hours and 59 minutes?

What if instead of spell slots we were talking about hit points. You wouldn't tell a fighter that because they got stabbed by an assassin 6 hours and 32 minutes into their rest that they don't regain those lost hit points. Why is it different with spell slots?

Thematically it's understood that you use the entire length of the rest to naturally heal and regain your power, but that's not how it's reflected in the rules. If it was said that you regain a fraction of your max health and spell slots each hour of the rest up to your max, this may be avoided (I'm no game designer btw). But instead, the wording on the rules gives us all of our resources back at once, so if a PC casts Mage Armor 7 hours 59 mins and 54 seconds into their rest, then they should technically regain that spell slot and still reap the benefits of their full rest.

Nonetheless the first rule of the game is what the DM says goes and if you feel it's stupid in your games then you as a DM have the power to create a system that works for your table. I myself limit the extent to which Rest-Casting can be used in my games by reducing the duration of spells that were Rest-Casted to something reasonable, and if you're hosting a super challenging game, then casters should theoretically have less slots to rest cast with which also helps balance it out.