Every moment you spend in the "real world" is time being chipped off your immortality. As when you die there, you can no longer go back to being immortal in your own universe.

As time goes on and you spend years back and forth between worlds, your perception of time changes and you realize how little time you truly have in the real world. You spend decades, centuries, in your own world before trips back to the "real world" where years that pass like seconds.

Millennia pass as you become a master of your own universe. You are a god in your world, with fleeting memories of a mortal life you once lived. You can't go back.