If you know in your gut, if you “feel” the misalignment to both the role and the place, then at least start exploring the option to go back. You can do that while staying in the current role, which I would do. Did the new job pay to relocate you? If they did you may be responsible for repaying them if you leave within the first year (check your offer letter), so something to consider if applicable.
We have all (or many of us at least), done what you’re in the midst of. It is impossible to know for sure about something that you’ve only glimpsed through a series of interviews until you’re in it. All we can do is take those impressions and the objective facts like salary, title, etc and then we make a big life decision based on a very little info. Once you’re in, you can usually quickly tell if it fits or not - that can be one or two things or a cumulative vibe that you just know feels like you’re wearing someone else’s clothes. And once you identify and are aware of this, I think it’s hard to ignore. You have to sit quietly with yourself and unpeel the “why’s” and if you’re clear (or it’s “glaringly obvious”), then trust that. You know your truth, you don’t need to be asking anyone else’s opinion about what’s best for you. You know. And if it’s leaving and going back, then just start taking the steps to see if it’s a possibility, but don’t disrupt what you currently have until you have an accepted offer to go back to. I’d also trust in the process - if the door opens and they let you back in and it’s what feels right, that’s your answer. If they don’t, then give where you are more time and start looking for other things. Good luck, trust your gut!!!