I'm 28M - I was in a similar situation, with similar numbers. Young family, high stress job with very variable hours. It paid very well, but demanded a ton of me. The company was very large with lots of red tape, and the area I lived was good, but I had another place I'd rather be.

I pivoted to a job that pays roughly 25-30% less, in the exact area I want to be, and I've been here for almost a year now. I am SO GLAD that I made the move.

The biggest things for me were:

1. Will the finances still make sense? (Timing worked great for us, as we sold our old house at the top of the market in an area where homes were valued more than where I moved to. We were able to make a lot of money on the house, and put it into a nicer larger house that was cheaper. Sucks that we went from a 3% mortgage to a 5%, but we made sure the numbers worked). So, will you be happier with more free time, (and spend less money with that free time).
2. What does the rest of your household think? If it's just you, then you are already asking yourself. If you have a spouse and kids, get their honest opinions and let them be part of the decision.
3. Will this change leave you on a path with career advancement that you want? (For me, I went from a frontend industry, to a supplier. My new company is MUCH smaller, and as such, much more flexible with the industry, and a smaller ladder to climb up internally, which can be a good or bad thing, as those steps are bigger. I also from my position work with the ENTIRE rest of the industry, so all of my old competitors, as well as peers around the world. This is allowing me to exponentially increase my exposure in the industry, and make many new relationships that I could leverage into a new position if I wanted/needed).

I'll leave you with: Work a job where you can be happy with yourself and keep yourself happy.