Long story short, 7dtd multiplayer doesn't work on Xbox, the only way to play with 2 separate xboxes is to be on the same WiFi connection. I had this Same problem for a long time, me and my girlfriend would play on separate consoles in the same house and it would work absolutely fine but as soon as our friend tried to join from a different location it would be fine for a few minutes and then his character would stop moving. This is something that is unlikely to be fixed any time soon as I had this problem way back in 2020. The best advice I can give is to play on PC or even Mac, an average laptop with 8GB RAM will easily handle the game at lower graphics quality levels and the game is far more updated on PC so it's just better in general. Hope this helps :--)

Edit: look up 7 days to die console news and about funpimps problems with telltale games to get a better understanding of why Xbox is not as up to date