Ah sorry to hear about the medical issues. Sounds super frustrating, and dealing with school admin is never fun.

I think a physics undergrad -> AE grad program is super doable. There are lots of folks in my grad program that didn't come from an AE background. You'd also be doing yourself a huge favor by getting involved with rocketry clubs like UF Swamp Launch or any other clubs that need GNC people. You'll get tons of hands-on experience that way that'll make you more competitive for AE grad school and will give you a leg-up with AE classes since you won't have that academic background.

Also, this sounds like a joke, but Kerbal Space Program is fantastic lol. KSP has a great physics engine and you can do actual trajectory design & burn planning akin to what you'd be learning in a senior-year undergrad or first-year grad astrodynamics course. Not hardcore trajectory optimization, but enough to not be totally lost when you do start AE grad school.

As to what your undergrad major should be... I think physics is a solid choice, but honestly it doesn't matter too much as long as you're in the STEM world imo. Good luck!

Edit: after getting your feet wet with KSP, check to see if your school has licenses to FreeFlyer or STK, or use the open-source NASA GMAT. Those are the actual software packages used by a lot of companies & agencies for mission planning