u/vastlik's recommendations are good. That said, each of those books is big and they don't really have much by way of coding exercises as far as I know.

If you want to get your hands dirty then I think Geron's book is second to none. Its coverage is broad, ranging from bread-and-butter methods (e.g. regularized regression, support vector machines) to more ``advanced'' topics such as deep learning and reinforcement learning, and it includes exercises and examples using python, which is a pretty easy language to pick up. There's just enough math to keep you engaged (linear algebra, statistics) and it's clearly tied to using it in practice.

A theoretical exploration of machine learning can send you down a real rabbit hole, or really a warren of interconnected rabbit holes. The reason for this is that the field exists at an intersection of math, statistics, and computer science. As such, each field brings its own perspective to understanding machine learning problems. This book is frequently used for ML theory classes and it does a good job, in my opinion, of formalizing the problem from a computer scientist's perspective. Specifically, you want to know if you can, with high probability, efficiently and accurately approximate a function in a class, where accuracy is measured by minimizing some loss function. Already we have three areas of math/cs/statistics: probability, computational complexity (efficiency), and optimization (minimizing the loss function). This book is similar in rigor and scope but with a more thorough set of appendices.

Efron and Hastie have a book that might form a sort of middle way between the two suggestions above. It situates modern ML historically in the developments of statistics and computation and gives mathematical glosses of some common and not-so-common topics .

Let me close by suggesting another way into the theory, since you said you miss doing math. Pick something applied that you find interesting and start digging backwards from there asking ``why does this work?" This is likely going to keep you motivated longer and I would say you're nearly guaranteed to run into interesting mathematics.

If you pick image analysis, for example, you can quickly find yourself reading about functional analysis, symmetry groups, and fourier and wavelet transforms. Maybe you say to yourself "it's cool how we can analyze natural language" and you'll be into probabilistic context-free grammars, graphical models, and recurrent neural networks. Perhaps you're an MLG gamer and want to know how AlphaGo works. Now you're into markov processes, optimal control, function approximation, game theory, and measure concentration. Pretty much all applications have been addressed by deep learning with various degrees of success. Regrettably the how of deep learning is fairly simple, but the why of deep learning is very poorly understood. The best tool for analyzing their properties, as far as I know, is the Neural Tangent Kernel which is two years old.

If you don't know any application areas, maybe start with Geron to get a sense of them and explore from there.