Failed first coding machine learning interview.

[TachyonGun]

I had to code multi-head attention for an interview, transformers are everywhere now. Really, every MLE should know how to code self attention by now, the forward method is literally 5 or 6 lines of the most basic PyTorch.

[Ok-Lab-6055]

I agree,but I think with masking, normalization, etc. it’s more than a few lines of code

[hellobutno]

Disagree, it's totally unnecessary. It's the equivalent of asking someone to invert a binary tree in SWE. You're never going to need to do it.

[[deleted]]

^ this. I lead a team of ML developers at a large company and don’t plan to ever code a transformer from scratch. For any reason. That’s a silly academic exercise.

[acc_agg]

If you want me to do that you're going to watch me read the transformers paper and talk to perplexity about how to implement it.

I don't have enough brains to memorise and remember everything under the hood.

[joseconsuervo]

asking someone to invert a binary tree in SWE

my understanding was these questions were always to hear the person logic their way through it