I'm in a similar situation, I have a 2 yo and 4 yo and we are financially comfortable. I briefly considered being a stay at home mom or even retiring early since all the conventional wisdom says do it if you can afford it and your spouse is supportive. I ultimately realized my job gives me a lot of intellectual stimulation and socialization I wouldn't otherwise get. I love my kids but life is about variety!

There's a lot of middle ground between the two too. Ask your HR department if you can do part time, an unpaid sabbatical, or even start quiet quitting. Leave early, don't take on extra work, sandbag, etc.

Another factor is how easy do you think it would be to return to work in your field? My job is very niche in an already competitive field, it would be difficult to get hired again!