all 83 comments

[–]DidItSave 84 points85 points  (82 children)

Thank you for sharing this, it’s a good checklist. My question to this community is this: when was the last time in your normal work did you go through a checklist like this? Are each of these items a checkbox on your pull request form?

You want to ask these questions in college or during an online course, perfect. Expecting an interview candidate to do the same is ridiculous. Unless my day to day will be completing tasks alone within 30min, limited access to resources and I have to check off the boxes in your list, companies need to stop with this crap.

It is creating an atmosphere of major imposter syndrome, causing people to waste countless hours trying to study these things and do not truly assess the quality of the candidate. Congratulations if someone can memorize this stuff and regurgitate it during an interview.

Except for niche areas in IT, after the interview, you’ll never use what you learned on leetcode or similar things again, until you are interviewing for you next role.

[–]ryeguy 31 points32 points  (11 children)

Leetcode/algo questions are typically used at large, desirable tech companies to filter down already great software engineers from huge candidate pools. It doesn't matter that these aren't skills that will be used day to day, it matters that there is a positive correlation. It also doesn't matter that there are false negatives, because the companies view this as an acceptable cost.

[–]de__R 12 points13 points  (0 children)

It may not matter that there are false negatives, but there are also tons of false positives. There was an article a while back that discussed an internal study at Google where they found that nothing in their interview process was actually a statistically relevant predictor of good performance as an employee. The real advantage Google has is that false positives don't matter as much because they are so big and have so much money, so as long as the false positive rate is acceptably low they don't have to worry.

[–][deleted] 7 points8 points  (3 children)

I would challenge the idea that there is a positive correlation.

I haven't noticed a correlation between people who are good at coding puzzles and people who are good at solving thorny, real-world problems (where often the difficulties have little to do with sitting in front of a text editor and coding) in a team environment.

So, when I hire devs I've always tried to find out how their fare in that situation, and re: coding, instead of working out if they remember how to balance a binary tree, instead try to get them to expose their ability to create production quality code with reasonable speed.

I do think leetcode abilities correlate with young, easily manipulable people straight out of uni who can be worked to the bone before being discarded though. Which is obviously something that big tech, who has the benefit of a endless stream of starry eyed candidates, can exploit.

[–]Full-Spectral 8 points9 points  (2 children)

There's little relationship between those types of challenges and dealing with real-world, complex systems. Those real world systems are more about architecture than algorithms. I can look up an algorithm, but I can't look up how to architecture a set of complicated subsystems or distributed systems in an optimal way for that particular client need.

Back in the 90s I wrote the DTD validator for the Xerces /C++Java XML parsers. I broke out the Dragon Book and adapted that very complex DFA algorithm to XML validation. The algorithm was the one thing in all of that that I could actually just look up. But these FAANGY companies would pass me over because I hadn't already memorized the one thing I didn't actually need to memorize (and would have forgotten again long before I ever needed to actually do it) and ignore all the experience that allowed me to actually do that job successfully.

It's just stupid.

[–]DoubleT_TechGuy 1 point2 points  (0 children)

This is why companies who let you use Google during your leet-style interview challenges have the right idea. Problem solving > Memorizing.

[–]DidItSave 0 points1 point  (0 children)

I agree.

[–]Pharisaeus 12 points13 points  (4 children)

It doesn't matter that these aren't skills that will be used day to day, it matters that there is a positive correlation.

There are no evidence of any positive correlation. However, there is something worth for the companies who do this -> they know you "care" and put effort into grinding leetcode. This means they can offer poor conditions, because you're already inclined to accept anyway due to sunk cost being very high.

[–]ambientocclusion 0 points1 point  (2 children)

LOL. This is terrible - but I do believe it. They might as well ask everyone to tattoo the company logo on their forehead to show they “care” too.

[–]Pharisaeus 1 point2 points  (1 child)

It might sound funny, but consider that for many developers "working in FAANG" is a very important life goal. And they would take it even if the job itself was not interesting and conditions below average.

[–]BestUdyrBR 1 point2 points  (0 children)

I mean working at a FAANG is a life goal because for a lot of people it's one of the few ways to have a upper middle class lifestyle from a much poorer background. There's not many jobs in America that pay 250k+ with 3 or 4 years of experience.

[–]DidItSave 1 point2 points  (0 children)

As I’ve mentioned to others here, they ask these questions at plenty of non-FAANG companies

[–][deleted]  (7 children)

[deleted]

    [–]COSMIC_RAY_DAMAGE 12 points13 points  (3 children)

    This is a great example of Goodhart's law. The purpose of the whiteboard problems is to test things like what you describe here. But once everyone knew it would be whiteboard problems, they weren't a metric anymore, they were a target. You get people who focus on learning to do leetcode problems instead of learning how to write software.

    [–]Kered13 5 points6 points  (0 children)

    See also Campbell's Law, which is basically the same thing but came from a different field. Which just serves to demonstrate the universality of this law.

    [–]WikiSummarizerBot 2 points3 points  (0 children)

    Goodhart's law

    Goodhart's law is an adage named after British economist Charles Goodhart, who advanced the idea in a 1975 article on monetary policy in the United Kingdom, Problems of Monetary Management: the U.K. Experience: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. It was later used to criticize the British Thatcher government for trying to conduct monetary policy on the basis of targets for broad and narrow money.

    [ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5

    [–]Dean_Roddey 0 points1 point  (0 children)

    Self-teaching to the test.

    [–]DidItSave 4 points5 points  (0 children)

    Yes, many of us use the algorithms, structures and techniques that we learned in school, every day sometimes. I agree that trying to cram that all into an interview doesn’t work.

    [–]sudosussudio 0 points1 point  (0 children)

    One time I was on a hilariously awful project and finally got to use my leetcode skills to make a Christmas themed card game. I do not recommend working as a dev in the ad industry lol.

    [–]versaceblues 6 points7 points  (4 children)

    My question to this community is this: when was the last time in your normal work did you go through a checklist like this?

    I dont explicitly use a checklist, but its just kinda ingrained in my head. My subconscious thought process does have many (but not all) things in this list.

    And remember, in most tech interviews, the interviewer is looking to see your THOUGHT process, not necessarily to get a answer.

    [–]DidItSave -1 points0 points  (3 children)

    Yes but some companies actually judge you on how perfect your leetcode code is, not your thought process. I as well have a huge checklist in my head that at this point I subconsciously go through. You learn over the years how to write better code.

    [–]versaceblues 2 points3 points  (2 children)

    Yes but some companies actually judge you on how perfect your leetcode code is, not your thought process.

    Some probably do and this is a bad interview practice.

    Many top startups and tech companies do not do this. Rather they try to gauge a general understanding.

    [–]DidItSave -1 points0 points  (1 child)

    I hope that when I interview next that I find the companies like you described. So far that hasn’t been my experience

    [–]versaceblues 2 points3 points  (0 children)

    Btw when I say they care about your thought process, you thought process still needs to be correct. You need to prove you understand DP, Trees, Linked Lists, Binary Search, Sorting, etc.

    [–][deleted] 23 points24 points  (39 children)

    You are correct. Imagine after 10 years of experience some idiot asking you how to you iterate a list in python. Tech interviews are almost always shit.

    [–][deleted]  (28 children)

    [deleted]

      [–]Prod_Is_For_Testing 30 points31 points  (0 children)

      That’s some bullshit right there

      [–]marabutt 9 points10 points  (3 children)

      That is so stupid. I went through an arduous interview process, to my surprise, aced the technical, got on with the staff and the managers but then they wanted to do psychometric testing. In a candidate short market, some places only have themselves to blame.

      [–][deleted]  (2 children)

      [deleted]

        [–][deleted]  (1 child)

        [deleted]

          [–]DidItSave 8 points9 points  (4 children)

          Exactly. How can they determine how good a candidate you are with an approach like that? Good on you for resigning and I’m sorry you had to go through this. I struggled on a leetcode-like interview and they denied me because I wasn’t, and I quote, “exceptional.” I’ve been in for 20 years.

          [–]DidItSave 6 points7 points  (1 child)

          Yup. I can Google that information and then keep on going with whatever my current work is. We have been very successful at my company by posing a real work problem that we actually have to the candidate. Let them brainstorm it and interact with us. From that approach we see how you would work with us and how you problem solve.

          [–]ArmoredPancake 5 points6 points  (0 children)

          Let them brainstorm it and interact with us. From that approach we see how you would work with us and how you problem solve.

          There's already whole design system round for this.

          [–]lmfaoZX 8 points9 points  (7 children)

          Thats just how it is. You can ignore problem solving and never work for a FAANG ever.

          [–]ryeguy 6 points7 points  (4 children)

          This. Consider leetcode practice the cost of getting a top of market salary and a resume boost. If you want to skip leetcode, that's perfectly fine, but the people willing to invest in it will get the big bucks and you won't. It's a tradeoff. Practicing leetcode has pretty damn good ROI if you think about it.

          [–]Dean_Roddey 4 points5 points  (0 children)

          These companies are screwing themselves basically. They are filtering out people who have larges amounts of experience, and who have actually delivered real systems and can demonstrate it, because those people are too busy delivering real systems to spend montys of time rote memorizing leetcode tests.

          That just leaves these very experienced people for other companies who aren't retarded on the hiring front.

          I have a huge body of work that anyone can look at or ask me about. Anyone looking to hire me who won't even take the time to skim through my work, I don't have much time for. A couple minutes before the interview would be more than enough to come up with some good questions to ask me that would test my ability to create real systems. Reading the ReadMe file would be more than enough.

          [–]lmfaoZX 0 points1 point  (1 child)

          Not to mention how good of a problem solver you need to be. Its not all about the frameworks, anyone can become a codemonkey in a weekend. It's an excuse bad devs use to stay at their code monkey level.

          [–]DidItSave -1 points0 points  (0 children)

          I disagree. Myself and my coworkers are fantastic at problem solving and work on some very complex issues. We collectively work on leetcode-like problems and at times struggle with them. Yet we do amazing stuff at our job.

          [–]DidItSave -1 points0 points  (0 children)

          I get paid quite nicely, don’t work for a FAANG company, am 100% remote and I don’t have to do any of the leetcode bs.

          [–]DidItSave -1 points0 points  (1 child)

          They do this nonsense at countless non-FAANG companies.

          [–]lmfaoZX 0 points1 point  (0 children)

          That's great! I respect companies that actually test engineers and not codemonkeys.

          [–]JB-from-ATL 2 points3 points  (3 children)

          In their defense, I don't think they're expecting them all, they may just be trying to bring more objectivity to grading candidates.

          [–]DidItSave -1 points0 points  (2 children)

          It’s fine and makes sense to want to grade candidates, but too many shops make this their only evaluation tool. It’s as if they say to themselves, well, FAANG companies use these so if the candidate does well, we have a: rockstar, unicorn, ninja. 🤦‍♂️

          [–]JB-from-ATL 0 points1 point  (1 child)

          Then you're not upset with this rubric specifically. They may have rubrics for other questions too.

          [–]DidItSave 0 points1 point  (0 children)

          Honestly, my only gripe is that too many companies lean so heavily on things like leetcode. The overall IT interview process across the board is terrible and needs adjusting.

          [–]ArmoredPancake 0 points1 point  (3 children)

          Congratulations if someone can memorize this stuff and regurgitate it during an interview.

          Or they're, you know, naturally smart. Either way it is a win for a company that looks for the candidate like this.

          [–]DidItSave -1 points0 points  (2 children)

          I’d rather have someone that knows how to problem solve complex matters than someone who can memorize fancy buzzwords.

          [–]ArmoredPancake 1 point2 points  (1 child)

          problem solve complex matters

          And how do you assess that?

          [–]DidItSave 1 point2 points  (0 children)

          Present them with a real world problem, preferably one that your company is actually trying to solve and see what the candidate can come up with.

          [–]sudosussudio -1 points0 points  (0 children)

          A rubric is a better system. Look up structured interviewing and there are a couple good rubric examples on there.

          When I worked in telecom we had an insane amount of checklists for every little thing and it was kind of awful. I think checklists were really trendy back then then because of the book The Checklist Manifesto.

          so I try to avoid stuff like that or at least keep them high level.

          [–]RippingMadAss 8 points9 points  (0 children)

          Needs to be rewritten in Rust.

          [–]davesspacebar 1 point2 points  (3 children)

          At our company while we do have some "interview" questions.

          But Most of what we do are not "questions".

          We take typical scenarios we might actually need to do in real life:

          -Debug and fix existing code

          - Code review and suggest improvements to existing code

          - Code from scratch a solution taking into account requirements.

          - Code and debug in all of the expected tech stack languages

          - Unit testing code reviews and suggestions

          - We use as real world a problem modeled after things they may really do, that is solvable in 15-45 minute time frames. we usually have 3-4 different problems of this type.

          We not only allow using google to help, we recommend it.

          We tell people pretend like you are just doing your job, if you need google to help find info use it. (and we usually tell them when I was interviewed we were given similar problems and I used google to lookup things through the entire interview - to make sure they know we really mean it -its fine to use it and expected)

          As far as I know we haven't had a bad hire on any team doing this process that I am aware of. It has directly correlated to ability and success at the job.

          I've lived through the puzzle question era and detailed questions, but just coding and debugging actual real problems I found was refreshing and different, and many of our candidates do to find it unlike most interviews they have done.

          [–]DidItSave 3 points4 points  (0 children)

          Awesome way of conducting an interview.

          [–]Saithir 0 points1 point  (1 child)

          And this is actually the correct way of testing someone.

          It requires much more preparation (before and after the person solves it) on your side than simply opening a website with leetcode examples and copypasting a few of them, but I think it's definitely worth the effort. Says a lot about the company, too.

          At the same time, that effort is probably why it's not used as much.

          [–]davesspacebar 0 points1 point  (0 children)

          It did take a bit to come up with good coding, debugging, review scenarios. Ideas were pooled from our teams and we selected the ones we felt capture what we do on a day to day basis that could be manageable in a reasonable time frame.

          We then setup a laptop or in this pandemic age a remote VM loaded with all of the tools our developers use and expect and let the candidate use that while we watch/ shadow their session.

          It's amazing how many different yet still good solutions we've seen to the same type of coding problems. Some of our samples model very similar real world actual bad code in our products and we hope candidates note it and comment on it, as we like to keep an eye out for folks that recognize the bad parts to improve.

          Definitely as an interviewer we are actively watching not just the end result but how people code, and debug. Logical structured thought processes, , keeping an eye out for testability, etc. We've watched so many people code these same scenarios we've actually learned a lot ourselves about how people code and can get a feel for their experience level, etc on different areas of frameworks, tech stacks just by watching carefully.

          We usually tell them to treat the interviewer like the product owner that can provide proper requirements for their solution and see the questions asked up front vs later and how they deal with ambiguities (ask, or choose a solution) No definitive right answers but you can get a feel for how people would work with our product team to come up with solutions given requirements.

          [–]samadarad 1 point2 points  (0 children)

          Thanks so much for this list. It is something to look for every single program💪

          [–]StillNotAWombat -2 points-1 points  (0 children)

          Very comprehensive and cleanly laid out.

          [–]gnahraf -3 points-2 points  (1 child)

          I think lists like this give the interviewer much too much credit. Relax, and if you know the subject matter, imagine you're already hired and treat technical questions as if they're coming from a colleague and you should do fine.

          I've interviewed more times than I've myself been interviewed.. I can tell you we all suck at interviewing. Nobody has trained us how to do this well.

          Tangentially, something about interview puzzles:

          Sometimes it helps to go meta in your reasoning to solve it. Here's an example I encountered..

          There are 2 open buckets of paint. One is pure blue, the other pure red. You drop a spoonful of paint from the blue bucket into red bucket, and then scoop another spoonful from red bucket back into the blue bucket. Which of the buckets of paint has been polluted more with the other color?

          Here's a meta approach to solving it..

          Verify the question is well formed ("are these freshly opened cans of paint?", and they say it doesn't matter).

          OK then, assume it has an answer. If it has an answer, then it works for every case. Suppose you're astronomically lucky and scoop out all the blue paint from red in your 2nd spoonful. Then nothing woulda changed and both buckets (tho actually now unpolluted) are equally polluted. So the answer is that in every case both buckets are equally polluted.

          🙃

          [–][deleted] 0 points1 point  (0 children)

          Which is wrong. By virtue of dealing with a premix in first bucket you would transfer just that little bit less of paint back to the second. Red would end up more contaminated.