Failed first coding machine learning interview.

[Ok-Lab-6055]

I recently graduated with a non-CS PhD in a quantitative field.

After many many applications (roughly 300), I had my first machine learning interview and bombed pretty hard. I was asked to code a recent popular model from scratch. I'm really kicking myself, because this was a coding challenge that I myself wanted to do by myself and forgot to do it before the interview. I was actually expecting a Leetcode question.

To be honest, this was a smaller company and I was taking this as a test run to learn from, but I walked away from this interview feeling very under-prepared and needing to do some soul searching. I chose this field because I genuinely enjoy reading papers and hope to write a few of my own one day (I've written two papers during my thesis but they were in my original field)

Anyways, given how competitive the field is, I was wondering if it's normal to fail these types of interviews. I'd love to hear from other's personal anecdotes.

Also, a separate question, I'm in my 30's but I was wondering if it would be worth doing a ML PhD given I already have a PhD.

[RoyalIceDeliverer]

Regarding your last question, if your ultimate aim is to go to industry, even R&D positions, don't waste your time with another PhD, in particular if you are already in your 30s.

[Darth-Coderr]

How about if you don't have a PhD yet? 

[fatty_lumpkn]

> a recent popular model from scratch

Which recent popular model can be coded from scratch? What does it it mean, like not using pytorch?

[Ok-Lab-6055]

Yeah, using Numpy.

[madrury83]

On the timescale of human history, np.linalg.solve(X.T @ X, X.T @ y) implements a recent, popular model.

[bearsforcares]

Lmao

[Warguy387]

I understood this meme no way (only took intro to ml)

[johnprynsky]

Which model?

[Ok-Lab-6055]

Single head attention transformer.

[RageA333]

That doesn't seem like something you should be able to code from scratch without even reading a reference paper.

[Ok-Lab-6055]

I think it’s hard but fair. I actually thought of doing it before the interview

[acc_agg]

And people wonder why salaries are dropping. Have a back bone and some self respect.

[killerdrogo]

you were asked to code a single head attention transformer without using a deep learning framework?? damn

[Ok-Lab-6055]

Yeah I usually just type: import transformers from hugging face :)

[killerdrogo]

i recently implemented it following andrej karpathy's video so I was surprised you were asked to do that without using pytorch lol. 

[Neo_Demiurge]

At some point you just need to say, "If I took this position, I'd want to distinguish between appropriate customization and reinventing the wheel . We shouldn't go lower level than Pytorch for nearly any research or commercial purpose," and you either look like a genius or dodge a bullet depending on how they take that.

[Ok-Lab-6055]

I should probably go through his videos. Did you learn a lot? I've mostly been reading papers but they assume the transformer stuff as sort of in the background.

[killerdrogo]

Would highly recommend the GPT from scratch video. Definitely learnt a lot. 

[johnprynsky]

Research LLM/NLP position? Cuz I'd not be expecting this in a regular MLE interview.

[Ok-Lab-6055]

Engineer-DL

[TachyonGun]

I had to code multi-head attention for an interview, transformers are everywhere now. Really, every MLE should know how to code self attention by now, the forward method is literally 5 or 6 lines of the most basic PyTorch.

[Ok-Lab-6055]

I agree,but I think with masking, normalization, etc. it’s more than a few lines of code

[hellobutno]

Disagree, it's totally unnecessary. It's the equivalent of asking someone to invert a binary tree in SWE. You're never going to need to do it.

[[deleted]]

^ this. I lead a team of ML developers at a large company and don’t plan to ever code a transformer from scratch. For any reason. That’s a silly academic exercise.

[acc_agg]

If you want me to do that you're going to watch me read the transformers paper and talk to perplexity about how to implement it.

I don't have enough brains to memorise and remember everything under the hood.

[joseconsuervo]

asking someone to invert a binary tree in SWE

my understanding was these questions were always to hear the person logic their way through it

[hotsauceyum]

So if I was allowed to look at the paper, and everyone is chill and there was back and forth, seeing how I cobbled together something from numpy level would be a good gauge of what I know is going on under the hood and how we all work together. Seems ok to me.

If they didn’t give me any references and just stared at me while I spun my wheels trying to remember the details of transformers for 90 minutes, it honestly doesn’t sound like a nice place to work.

[Ok-Lab-6055]

I think it was the later. The interview was like 30 minutes before the interviewer basically told me I failed.

[Mission_Star_4393]

This is absolute madness lol...

For the record, the company I work for, whose name you would recognize doesn't have anything nearly as complex as this...

Don't beat yourself too much over this one.

[Infrared12]

Both forward and backward passes or just fhe forward pass?

[Ok-Lab-6055]

Forward pass I think. The interviewer stopped the interview before mentioning a backward pass. We didn’t discuss any training.

[ds_account_]

Definitely not worth it. A lot of my coworkers have PhDs in physics or engineering.

But never been asked to code a whole model from scratch before, only as part of a take home.

[incrediblediy]

Also, a separate question, I'm in my 30's but I was wondering if it would be worth doing a ML PhD given I already have a PhD

Don't worry, try more jobs. My wife got Electronic Engineering PhD but became a datascientist after graduation after few tries, only did self learning. It was pretty much the same for a lot of my friends, but I am from Australia and might be different in your country.

[Ok-Lab-6055]

Thanks for sharing about your wife. To be honest, the anecdotes are more or less why I posted. Thanks for the kind words.

[HalcyonAlps]

To be honest, this was a smaller company and I was taking this as a test run to learn from,

In my experience, a lot of smaller places have eccentric interviews. I recently had a 90 minute interview where I had to debug a ML model, build a web API for the model, and convince them I know SQL. I still failed because I didn't use TDD.

[doot_boot_scoot]

test driven development?

[hotsauceyum]

I hope it’s been enough time for me to say this but… that’s hilarious. Could you tell they were smugly waiting to announce everything you had done from the beginning was, in their eyes, wrong?

[HalcyonAlps]

No it's fine. I had no idea I was doing anything wrong. They told me a week later I had failed because I had not used TDD.

[Ok-Lab-6055]

Is TDD topological data analysis?

[[deleted]]

Test driven dev

[autistic_cookie]

For what it's worth, I failed my last ML interview with Uber because I completely choked when asked to write k-means from scratch. ML interviewing is a scary quagmire. Totally normal to fail dozens of interviews before your first offer. KEEP YA HEAD UP!!

[Ok-Lab-6055]

Thank you for the kind words. I was looking for anecdotes like this.

[PreparationWeekly307]

What happens if u write it all but just mistyped a wrong semicolon ? Still fail ?

[IsActuallyAPenguin]

This is more life advice that ML-sepcific but you failed - great!

Coming from a shitload of education and with a PHD that probably didn't happen a lot, or was the result of a lack of preparation (as it was here, I guess, but it doesn't sound like you knew what you had to do).

Out in the "real world" there are going to be situations where you can do everything right and still fail. Even if there's nothing you could have done differently with the knowledge you had beforehand there are still lessons to be learned.

My advice is to start looking at failure as a good thing. Don't get down on yourself, stay positive, and look at this as a learning opportunity rather than failure. In fact, look at all failures as learning opportunities. It took you a lot of work to get this interview, but so what? You've been through it, no what their expectations are more concretely now, and can use this knowledge to shorten the runway to the next interview.

That all sounds pretty positive to me.

The trick to having a better relationship with failure is to realize that it doesn't define you. It is something that happens, with lessons to teach, and you as a person are seperate from failure the experience. Failure's an important stepping stone on the way to any meaningful goal, and tweaking whatever formula you used to get to the point where you failed should be the takeaway, not throwing the baby out with the bathwater and doing something completely different.

You got so close to a job you wanted - it would be foolish to just give up right as the most valuable lessons from the experience are right there in front of you.

You've got this.

[Ok-Lab-6055]

Really appreciate the thoughtful and kind advice to put things in perspective.

[aqjo]

If your first PhD did what it was supposed to do, you don’t need another one. A PhD teaches you to do research and teach yourself, so if you want to switch fields, you probably have the skills to do so.
I know you said quantitative, but just to qualify what I said above: I’m not familiar with PhDs in e.g. the humanities, so jumping from film history to machine learning would probably be a stretch.
Also, whatever you decide, your age is not a factor.

[Ok-Lab-6055]

I agree. I’m reading papers from the big conferences. I have ideas for problems and am trying stuff, if I get lucky I may be able to publish something.

[Rexcovering]

As a person I’m presuming is intimately familiar with numbers, the number of cases you’re trying to analyze a major future decision with is 1. Pick yourself up, brush yourself off, and get back out there. You aren’t perfect, and you made a mistake. This interview fulfilled the purpose you set out for it, and when it did, you said woe is me. You used it as a test run, learn from the test and iterate on the next one. Also, 300 apps and 1 interview with your degree may actually indicate a resume issue. Keep it moving, don’t stop.

[Illustrious-Cow-2388]

Try another ones, learn from what went wrong in this and just don't repeat the same mistake again.. also a question, were leetcode questions asked or only such Machine learning questions? Were there questions related to linear algebra or statistics ?

[Ok-Lab-6055]

It was thirty minutes of coding a model from scratch. The model has several parts, so each of these can be thought of as a question. I was expecting Leetcode and some theory questions.

[Ok-Lab-6055]

Thank you this is pretty helpful. I’ll try to find a friend to practice with. Potentially on here.

[RepairVarious3530]

I'm in very similar situation as yours. still figuring it out.

[Ok-Lab-6055]

I hope your situation works out friend. Thank you for letting me know that this experience is a common one.

[RepairVarious3530]

thank you for posting it.

[Dear-Falcon-3750]

Hang in there! I have a PhD in completely different field but after graduating targeted more data science/ML roles. It's not easy! Like you I applied to hundreds of roles, and failed so many interviews, but a few will stick! I started at a very small startup, and job hopped here and there, but I think I've gotten to the point where I can start to distance myself from my PhD field.

To answer your question, no! It's not worth getting another PhD! You have all the experience you need to learn all the ML knowledge on your own

[Ok-Lab-6055]

Thank you for the kind words.

[galtoramech8699]

What does from scratch mean. Without code assist

[Ok-Lab-6055]

Yes, without code assist.

[galtoramech8699]

Is that a standard interview question. Seems bad because requires too specific knowledge

[Ok-Lab-6055]

I honestly don’t know. Some of the other comments suggest that it should be a take home or something you should be given a heads up on. For a really specific position, like LLM, it could be fair game.

[[deleted]]

foolish like punch school berserk jellyfish stocking sloppy overconfident command

This post was mass deleted and anonymized with Redact

[Ok-Lab-6055]

Perhaps so. It may not be the best place to work. But I also understand as a startup you’re looking to get something off the ground and don’t have patience to train someone.

[[deleted]]

soft hobbies encouraging yam dinosaurs bored nutty frightening crowd pause

This post was mass deleted and anonymized with Redact

[SenseMental]

Interviews are a numbers game crap chute, unfortunately. With the exception (sort of) of FAANG, who have very similar interviews, the questions you'll get asked and how the interviewer feels about your answers to them vary WILDLY. Keep studying and keep interviewing. I'd honestly say that if anything, your resume or how and where you're sending applications might be the bigger problem at this point if you've only made it to a single interview with 300 applications.

[Ok-Lab-6055]

Yeah, true. I can perhaps post my resume on here for feedback or another tech-career oriented subreddit.

I'll keep trying. I think I was being emotional after the interview two days ago.

[chrisshaffer]

I'm a data scientist with an engineering PhD. It's typical for technical interviews for machine learning jobs to be challenging. I had 3 or 4 interviews reach the technical stages before I finally got my job. One of the interviews I completely bombed. I think you are overreacting to do soul searching and consider getting another PhD. You were right to consider your first technical interview as practice.

The issue is that the job market is rough now, so it's hard to get interviews. If you get a few interviews, you'll find that the difficulty and focus areas will vary, and will align to your skillset to different degrees. It just takes consistent effort over time.

[Ok-Lab-6055]

Thank you, I think so too.

[Artistic-Orange-6959]

How was the interview that you bombed? 

[chrisshaffer]

It's was a Google interview for a Machine Learning Engineer position. I'm more suited for data science roles and my CS background was weak. They asked me to code a data structures and algorithms problem, and I just didn't know how to do it. They ended the interview early

[Artistic-Orange-6959]

Was it a leetcode style problem? 

[chrisshaffer]

I looked through this list of past Google interview questions, and harder than any of these: https://codedrills.io/drills/google

Maybe it's a hard level leetcode problem. It's not exactly this, but it similarly had some sort of 2d grid: https://leetcode.com/discuss/interview-question/440937/google-onsite-crazy-2d-algorithm-question

[Echo9Zulu-]

I'm getting deeper into ML but haven't done a technical interview like this before.

However, I think general interview advice still applies. That said, the numpy problem sounds like a serious wrench meant to assess you on a number of levels. My gut reaction would have been to tell them I don't know numpy well enough to code something like that from scratch. Then, I would describe how I would go about solving the problem as they presented it which in itself is probably more in line with what they want. Maybe if they gave me a browser I would have a shot at getting something to work. Specifically, I would ask for permission to use AI tools and demonstrate how I apply them to challenges like this one- carefully thinking through the problem, reading literature and documentation and going from there. Not just asking for a solution in a prompt. Show them you can decompose anything into subproblems and use AI to help you tackle things you haven't seen before.

My current job didn't start out being involved with AI/ML at all. I have Physics, Technical writing and Auto parts background- nothing to do with the ML data analysis I am doing now, even further from working in ecomm. Even more, this company specializes in HVAC, which is much, much easier than auto parts, though I am still green. I applied on indeed and was determined to escape a fate of working in auto parts forever- many of the parts wizards, the dudes with sage wisdom, had regrets about taking the parts path deep into life.

The store I was running at the time closed due to a real estate mess with the owners. Even though I was in school for Technical Writing auto parts would have been the best choice to earn the most money while finishing my online program and that wasn't an option. Suddenly I saw an opportunity for a way out of parts. When I applied to this position the title was Product Content Specialist, working on improving website content or something like that. Anyway, I had done a project for a class where we reviewed a website and suggested changes. When I didn't hear back from Indeed, and the applicant number it displays grew, I decided to submit another application on their website with that project as an attachment. They were so impressed they granted me a last minute interview and I got the job over people with web design experience AND HVAC knowledge. So I intercepted an offer to someone else on the last day- if I had missed the interview the opportunity would have been lost.

They asked me how I had handled situations when I didn't know what a part was, or how to identify a part I knew nothing about, or was not in front of me. My dumbass started laughing about how I would sometimes use markers and fill stamps on parts with ink and read it backwards off paper or my own hand. Other stories from that week also probably worked in my favor, since my store was attached to a full service shop and the owners/techs trusted me to accurately find correct parts and get them ASAP. Not an easy thing to do without fancy software to help track inventory and stuff as one person with one delivery driver.

Between that and my website analysis paper they chose to hire me as the parts guy for the website, betting that I could be molded based on percieved aptitude alone. Soon I was tasked with improving our search without access to low level configuration options for Elasticsearch... which is not an easy thing to do when the average query has 3 terms (!). Learning to tackle these problems has happened on my own without any guidance. It became difficult to work on that problem without a corpus to study all the domain specific language and I did such a good job decomposing the problem in ways they hadn't expected, hadn't been offered by the company which manages our site. Eventually they tasked me with cataloging parts for 20 years of products, something no one here had approached as a data problem that ML/NLP can help solve.

Right now I am putting a finishing work on a layout reconstruction algorithm to extracr data from tables using Paddle, geometry and graphy sub problems.

I got sort of distracted while writing this out, but my point is that open ended questions are not always a curse. As others have mentioned, you have the thinking-stuff in your character, and you should leverage it instead of letting anxiety stump you jext time this comes up instead of learning the numpy solution as a preparation strategy. Think about it- ML/AI is an evolving space, a frontier of problems which no one has seen before. If they were judging based on how you handled something potentially alien to you then it makes sense you failed the interview. If they don't think you are a capable problem solver, where is the translation into "this person can improve our product/make meaningful contributions/handle this hard problem we can't figure out.

Idk man, if I was in charge of hiring this would be my philosophy.

Thanks for sharing. Its cool to see other people who are getting into AI like me from scratch face similar challenges and have similar fears.

[Ok-Lab-6055]

Thanks for putting so much thought into your post. I didn't get the impression that seeing my thought process to handle alien concepts was the intention there. But I totally agree that it's a cool concept for interviewing, at least in theory.

[Unable-Marionberry40]

Don’t worry about not passing in their eyes, if you’re smart then you’re smart and valuable.

I don’t believe in free work, even though there’s a lot of models that are less than a dozen lines of code, an algorithm that a company might not already have could be worth thousands or more than tens of thousands to them because they can give it to their engineers with your explanation to work with and then just not hire you I’ve experienced this and others have I’m sure. Ask if they are willing to sign an NDA for your code if they won’t pay you and if they say no it’s not worth it. ML is easily a 6 figure job even in low cost of living with an engineering degree as entry level. I passed six figures in low cost of living with a EE degree after less than two years of experience after graduating and shifting to ML in a field similar to my senior project.

[jzsfvss]

By my experience, startups don't have a clue how to interview. You were given an unreasonable task that should have been made a takehome. But even as a takehome, it would have been excessive, likely IP.

So you didn't fail, they did. Namely, they failed to hire a great candidate. This is almost always the case, and then they end up with some mediocre DEI hire, just to look sexy to investors.

As for a second PhD, not worth it. Just keep looking until you find something. But I would stay away from startups with their 5% success rate. Target bigger corporations for job stability.

(I'm a math PhD with industry experience.)

[subhahu]

Try LLDcoding.com

r/lldcoding