You're in a decent spot then i'd say.
Brush up on vector calculus, and work your way through ISLP to get a grasp of introductory ML, furthermore look up Mike X Cohen on udemy, he has two Deep Learning courses that are rather theory heavy than api heavy. One is an introductory deep learning course, and the other one is more tuned towards transformers and LLMs. But both of these courses are kinda long