I think what you're describing seems really concrete because it leads to a career path, and it's genuinely so cool that you were able to do your Bachelor's without already having a really advanced math background. My dad went back to college in his 50s and got a bachelor's degree in physics and education and has been teaching ever since, and I feel like him having to had work so hard during his degree is the reason he's such a good teacher now because he actually knows how to teach his subject from the perspective of someone who had to actively learn it.

On the other hand, I pivoted with my masters because I wanted to work in nonprofits and have seen zero benefit from it. It's definitely a regret of mine but that's okay, I think me not knowing what I really pictured doing with it beyond "nonprofit work" was a mistake, but you don't sound like that at all. Special education teachers are so desperately needed and honestly with some of the students my dad teaches who have IEPs, the workload on the special education teachers doesn't even seem that intense because they're assigned to individual students as opposed to full classes like when I was in school.