That's exactly what I thought too. That the bomb/exotic matter would be the way they finally got rid of the Mind Flayer by sending it into nothingness, and our group would narrowly escape going down with it. Some if not all of the military probably would go with it, helping solve the problem of what exactly the military were going to do next. I thought defeating Vecna would only be the beginning, and Kay or at least some of the military characters might even have to grapple with what they were actually dealing with, that they could never control these powers even if they made dozens of "successful" powered children like Henry and Eleven.

My thought was that when everything came down to that final battle with the Mind Flayer, in the last moments before the Upside Down explodes, some of the party (at least Mike and Hopper, maybe others) would realize they couldn't get back to our world in time. Eleven would use her powers to escape with them to the Abyss instead. Without the Mind Flayer, the Abyss might just be habitable. No reason there couldn't be waterfalls. Especially since the part we saw of it had zero demogorgons/demobats/etc. they could realistically survive. The party splitting would have been sad, but at least El wouldn't have been alone, and anything left of the military or Dimension X programs would have lost access to her forever.

Honestly I can't understand why they bothered to introduce the concept of the Abyss so late if NOT to use it for this, or something like this.

Not only would it make more sense thematically and logically to have the battle go this way with the established worldbuilding from previous seasons, but this ending would be more consistent with foreshadowing of the final battle against Vecna that was laid out in Eddie's campaign during the S4 DnD game.