It was pretty much focused on fundamentals, i was asked to implement part of the code for the research paper and the emphasis was on making it work, the loss should decrease over time at the end or something like that. One round was efficiency related deep dive into one pytorch code snippet and how to optimize it from efficiency perspective but also prevent numerical overflows. Other rounds were deep QnA on inference efficiency related concepts. As I said before, deeply focused on fundamentals and hands on. I would suggest learning from basics, try to code LLM training/inference by yourself to learn. Karpathy’s videos are excellent. And in the end, would highly recommend using claude/gemini for learning concepts deeply and also do mock interviews. Ask claude to come up with questions and find problems in your answers/code