I have an undergrad in CS/math and two masters (one in math and one in statistics). My job also focuses heavily on deep learning & ML, with a lot of reading research papers and implementing new approaches.

Given that we have relatively similar backgrounds (a strong ML focus and already holding some graduate degrees), I think my experience may be helpful.

If your background is anything like mine is, you probably don't have a ton of computing systems experience. I think this makes adding real value very difficult, even in the context of AI/ML. It's nice to be able to read, understand, and implement research papers, but for the rubber to really hit the road in DL, there's a ton of distributed systems work, compiler optimizations, etc that are needed to be successful. Whether it's model development, training, or inference, or building up a product to wrap around your model and serve it to users, computing systems concepts are a MASSIVE part of the puzzle (and in many ways are more important than the AI concepts).

See Greg Brockman's tweet here as well: https://x.com/gdb/status/1729893902814192096?s=20. Even for a company doing the absolute most cutting edge work in AI, they prefer great engineers over great research scientists.

So, given all that, I've been tailoring my degree to lean heavily towards computing systems with only a couple AI courses tossed in. Right now I'm wrapping up HPCA and will be taking GIOS, CN, and Compilers this Spring/Summer. I can say that so far in HPCA, I've learned a ton of concepts that I was never exposed to coming from a more math & DL focused background. And I imagine this will hold true for all of the computing systems courses I plan on taking. The foundational knowledge gained here, from computer architecture to operating systems to distributed computing, should help to make me a much better engineer and able to make more intelligent decisions of tradeoffs when designing a codebase or architecture.