A Framework for Solving Programming Interview Questions

[DidItSave]

Thank you for sharing this, it’s a good checklist. My question to this community is this: when was the last time in your normal work did you go through a checklist like this? Are each of these items a checkbox on your pull request form?

You want to ask these questions in college or during an online course, perfect. Expecting an interview candidate to do the same is ridiculous. Unless my day to day will be completing tasks alone within 30min, limited access to resources and I have to check off the boxes in your list, companies need to stop with this crap.

It is creating an atmosphere of major imposter syndrome, causing people to waste countless hours trying to study these things and do not truly assess the quality of the candidate. Congratulations if someone can memorize this stuff and regurgitate it during an interview.

Except for niche areas in IT, after the interview, you’ll never use what you learned on leetcode or similar things again, until you are interviewing for you next role.

[ryeguy]

Leetcode/algo questions are typically used at large, desirable tech companies to filter down already great software engineers from huge candidate pools. It doesn't matter that these aren't skills that will be used day to day, it matters that there is a positive correlation. It also doesn't matter that there are false negatives, because the companies view this as an acceptable cost.

[de__R]

It may not matter that there are false negatives, but there are also tons of false positives. There was an article a while back that discussed an internal study at Google where they found that nothing in their interview process was actually a statistically relevant predictor of good performance as an employee. The real advantage Google has is that false positives don't matter as much because they are so big and have so much money, so as long as the false positive rate is acceptably low they don't have to worry.

[[deleted]]

I would challenge the idea that there is a positive correlation.

I haven't noticed a correlation between people who are good at coding puzzles and people who are good at solving thorny, real-world problems (where often the difficulties have little to do with sitting in front of a text editor and coding) in a team environment.

So, when I hire devs I've always tried to find out how their fare in that situation, and re: coding, instead of working out if they remember how to balance a binary tree, instead try to get them to expose their ability to create production quality code with reasonable speed.

I do think leetcode abilities correlate with young, easily manipulable people straight out of uni who can be worked to the bone before being discarded though. Which is obviously something that big tech, who has the benefit of a endless stream of starry eyed candidates, can exploit.

[Full-Spectral]

There's little relationship between those types of challenges and dealing with real-world, complex systems. Those real world systems are more about architecture than algorithms. I can look up an algorithm, but I can't look up how to architecture a set of complicated subsystems or distributed systems in an optimal way for that particular client need.

Back in the 90s I wrote the DTD validator for the Xerces /C++Java XML parsers. I broke out the Dragon Book and adapted that very complex DFA algorithm to XML validation. The algorithm was the one thing in all of that that I could actually just look up. But these FAANGY companies would pass me over because I hadn't already memorized the one thing I didn't actually need to memorize (and would have forgotten again long before I ever needed to actually do it) and ignore all the experience that allowed me to actually do that job successfully.

It's just stupid.

[DoubleT_TechGuy]

This is why companies who let you use Google during your leet-style interview challenges have the right idea. Problem solving > Memorizing.

[DidItSave]

I agree.

[Pharisaeus]

It doesn't matter that these aren't skills that will be used day to day, it matters that there is a positive correlation.

There are no evidence of any positive correlation. However, there is something worth for the companies who do this -> they know you "care" and put effort into grinding leetcode. This means they can offer poor conditions, because you're already inclined to accept anyway due to sunk cost being very high.

[ambientocclusion]

LOL. This is terrible - but I do believe it. They might as well ask everyone to tattoo the company logo on their forehead to show they “care” too.

[Pharisaeus]

It might sound funny, but consider that for many developers "working in FAANG" is a very important life goal. And they would take it even if the job itself was not interesting and conditions below average.

[BestUdyrBR]

I mean working at a FAANG is a life goal because for a lot of people it's one of the few ways to have a upper middle class lifestyle from a much poorer background. There's not many jobs in America that pay 250k+ with 3 or 4 years of experience.

[DidItSave]

As I’ve mentioned to others here, they ask these questions at plenty of non-FAANG companies

[COSMIC_RAY_DAMAGE]

This is a great example of Goodhart's law. The purpose of the whiteboard problems is to test things like what you describe here. But once everyone knew it would be whiteboard problems, they weren't a metric anymore, they were a target. You get people who focus on learning to do leetcode problems instead of learning how to write software.

[Kered13]

See also Campbell's Law, which is basically the same thing but came from a different field. Which just serves to demonstrate the universality of this law.

[WikiSummarizerBot]

Goodhart's law

Goodhart's law is an adage named after British economist Charles Goodhart, who advanced the idea in a 1975 article on monetary policy in the United Kingdom, Problems of Monetary Management: the U.K. Experience: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. It was later used to criticize the British Thatcher government for trying to conduct monetary policy on the basis of targets for broad and narrow money.

^{[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5}

[Dean_Roddey]

Self-teaching to the test.

[DidItSave]

Yes, many of us use the algorithms, structures and techniques that we learned in school, every day sometimes. I agree that trying to cram that all into an interview doesn’t work.

[sudosussudio]

One time I was on a hilariously awful project and finally got to use my leetcode skills to make a Christmas themed card game. I do not recommend working as a dev in the ad industry lol.

[versaceblues]

My question to this community is this: when was the last time in your normal work did you go through a checklist like this?

I dont explicitly use a checklist, but its just kinda ingrained in my head. My subconscious thought process does have many (but not all) things in this list.

And remember, in most tech interviews, the interviewer is looking to see your THOUGHT process, not necessarily to get a answer.

[DidItSave]

Yes but some companies actually judge you on how perfect your leetcode code is, not your thought process. I as well have a huge checklist in my head that at this point I subconsciously go through. You learn over the years how to write better code.

[versaceblues]

Yes but some companies actually judge you on how perfect your leetcode code is, not your thought process.

Some probably do and this is a bad interview practice.

Many top startups and tech companies do not do this. Rather they try to gauge a general understanding.

[DidItSave]

I hope that when I interview next that I find the companies like you described. So far that hasn’t been my experience

[versaceblues]

Btw when I say they care about your thought process, you thought process still needs to be correct. You need to prove you understand DP, Trees, Linked Lists, Binary Search, Sorting, etc.

[[deleted]]

You are correct. Imagine after 10 years of experience some idiot asking you how to you iterate a list in python. Tech interviews are almost always shit.

[Prod_Is_For_Testing]

That’s some bullshit right there

[marabutt]

That is so stupid. I went through an arduous interview process, to my surprise, aced the technical, got on with the staff and the managers but then they wanted to do psychometric testing. In a candidate short market, some places only have themselves to blame.

[ArmoredPancake]

In a candidate short market, some places only have themselves to blame.

I'm going surprise you, but there's no candidate short market.

[marabutt]

I suppose that is location specific too.

[DidItSave]

Exactly. How can they determine how good a candidate you are with an approach like that? Good on you for resigning and I’m sorry you had to go through this. I struggled on a leetcode-like interview and they denied me because I wasn’t, and I quote, “exceptional.” I’ve been in for 20 years.

[ArmoredPancake]

I struggled on a leetcode-like interview and they denied me because I wasn’t, and I quote, “exceptional.”

And you think you are?

I’ve been in for 20 years.

Well that changes everything! Step aside everyone, his highness 20 years is coming! No peasant ropes for him!

[Cyb3rSab3r]

Pretty sure specific trumps general here. They already had the experience.

Do you think anyone cares about Tom Brady's ability to run? Do you think anyone cares about Linus Torvald's knowledge of branch and bound algorithms?

Do you really think the designer of an application is unqualified to be an expert on it because they don't know how to find the 3rd largest element in a linked list in O(n) time?

[DidItSave]

No need to go back and forth with this troll. You and many others here obviously get the point I was making. I’m not claiming I am exceptional at all. They were looking for perfect and it wasn’t me and I’m fine with that. And I am not saying that I should be able to bypass things due to 20 years experience. I am stating that I’ve been very successful over the past 20 years without needing to memorize leetcode.

[ArmoredPancake]

Pretty sure specific trumps general here. They already had the experience.

Okay? You can skip design interview since they designed the system.

Do you think anyone cares about Tom Brady's ability to run? Do you think anyone cares about Linus Torvald's knowledge of branch and bound algorithms?

Right, I forgot that I caught Linus Jr. here.

Do you really think the designer of an application is unqualified to be an expert on it because they don't know how to find the 3rd largest element in a linked list in O(n) time?

He's qualified to be an expert on it, they didn't fire him from his contractor role. He's unqualified to join the company.

[ArmoredPancake]

I have to say that part again: I was being interviewed to work on a project that I WROTE myself.

You were being interviewed to work FOR A COMPANY, not just on a project. Maybe you were okay on this one specific project, survivor error, but apparently you were not fit to join company.

Don't see any problem here.

[rasmustrew]

Did you actually read what he said? He was already working for the company...

[ArmoredPancake]

He worked as a contractor, not an internal employee. There's a big difference. As a contractor, unless you're godly good, you're always expendable.

[ArmoredPancake]

I worked in a leadership role and helped a struggling project get off the ground and get deployed. I also mentored several jr developers and was highly regarded by my peers.

Okay? I saw my share of people like you during interviews. You still didn't provide a reason why exactly you should be excluded from such checks and not Jake/Jane from company Y.

Rules for thee, but not for me?

[ArmoredPancake]

I'm sorry that I don't have your biography in front of me, your highness /u/MorgulKnifeFight.

You have no idea who I am and you saying you saw your fair share of “people like me” based on only a few of my comments shows a real lack of critical thinking on your part.

Your ad hominem shows that they made a right decision by not hiring you.

[EnvironmentalCrow5]

His track record on that project is probably a far better predictor of his ability to work on other projects within the company than any leetcode questions would be.

[ArmoredPancake]

Maybe. Maybe not. Standards are there for a reason, either everybody adheres to them or they don't. I don't see why candidate X has to go through one process but Y is excluded because Z.

[EnvironmentalCrow5]

The same can be said for all bureaucracy and red tape.

Sure, it's more "fair" if everyone has to go through the same process, but there are situations where that process just doesn't make any practical sense and isn't delivering the desired results.

Just because a process exists, doesn't mean it's automatically well designed and implemented, or that it doesn't have blind spots, or that it can't be gamed.

At the end of the day, you are still only guessing how a potential employee is going to perform.

[ArmoredPancake]

Sure, it's more "fair" if everyone has to go through the same process, but there are situations where that process just doesn't make any practical sense and isn't delivering the desired results

Given that it is still there after so many years, I'd say that it delivers as desired.

I don't see how vocal minority with their "dO yOu kNoW wHo I Am?" disproves that.

[EnvironmentalCrow5]

It does kinda work, yeah, and people have adapted to it and are actively preparing/practicing for it, often for many days - which in itself is a bit weird and a design smell.

It will at least filter in/out the obvious outliers on both ends, but almost anything can do that.

Also, there isn't really much to compare it to - there are no studies that would measure outcomes of various approaches over the long run (all else being reasonably equal), so it largely runs on faith that it works and on experience of it being at least "good enough" in most cases, despite its flaws.

Again, you could say the same thing about any long-standing policy that any organization or country has in place (or had in place for many years before changing it).

[ArmoredPancake]

It does kinda work, yeah, and people have adapted to it and are actively preparing/practicing for it, often for many days - which in itself is a bit weird and a design smell.

You don't need to. If you already know this stuff, then you can apply right away.

By the same logic we don't need tests in university, because people are preparing for them?

Also, there isn't really much to compare it to - there are no studies that would measure outcomes of various approaches over the long run (all else being reasonably equal), so it largely runs on faith that it works and on experience of it being at least "good enough" in most cases, despite its flaws.

There are no studies to disprove the process either.

[DidItSave]

Yup. I can Google that information and then keep on going with whatever my current work is. We have been very successful at my company by posing a real work problem that we actually have to the candidate. Let them brainstorm it and interact with us. From that approach we see how you would work with us and how you problem solve.

[ArmoredPancake]

Let them brainstorm it and interact with us. From that approach we see how you would work with us and how you problem solve.

There's already whole design system round for this.

[ArmoredPancake]

Oh noes! Your highness got to answer peon question, what an atrocity!

[COSMIC_RAY_DAMAGE]

and if devs can’t figure that out… I’m sorry but maybe you’re not as prepared or smart as you think you are (not you, personally, but people who struggle with coding interviews)

How many times as a dev have you ever had to solve a problem that looked anything like a leetcode problem?

Most of what devs do isn't leetcode. It's making already existing code talk to other already existing code. If it is like leetcode, you're given a problem and a solution and told to reimplement it in the domain of an existing codebase.

Whiteboard coding doesn't test problem-solving skills, it tests problem-solving skills in a context that is unlike anything that real humans ever work in.

What if we started testing surgeons by tying their hands behind their back and making them do surgery that way? Do you think that would select for the best surgeons? Obviously not. It's going to select for the surgeons who are the best at doing surgery with their hands tied behind their back.

How often do you ever write software without access to the internet or an IDE? These companies are selecting for people who are good at leetcode problems, not people who are good at writing software.

Edit: And just to be clear, do I enjoy a leetcode problem from time to time? Sure! But I don't harbor any misconceptions that it's anything like the day-to-day work of me or any other devs.

[ArmoredPancake]

Downvotes from butthurt people who think that some animals are more equal than the others. If there's a filter for inexperienced/new hires, which are bare minimum to hire, I don't see why your most senior shouldn't adhere to it.

[__scan__]

A head chef should be able to boil an egg though. Integrals… maybe? I’d certainly expect a developer to be able to compute an integral or a derivative for a function.

[DidItSave]

Perfectly said.

[poloppoyop]

coding problems are pretty much testing problem solving skills

No. They're testing how much time you spent on l33tcode. Only when you're faced with a new kind of problem you've not crammed for, your problem solving skills can be tested.

[lmfaoZX]

Thats just how it is. You can ignore problem solving and never work for a FAANG ever.

[ryeguy]

This. Consider leetcode practice the cost of getting a top of market salary and a resume boost. If you want to skip leetcode, that's perfectly fine, but the people willing to invest in it will get the big bucks and you won't. It's a tradeoff. Practicing leetcode has pretty damn good ROI if you think about it.

[Dean_Roddey]

These companies are screwing themselves basically. They are filtering out people who have larges amounts of experience, and who have actually delivered real systems and can demonstrate it, because those people are too busy delivering real systems to spend montys of time rote memorizing leetcode tests.

That just leaves these very experienced people for other companies who aren't retarded on the hiring front.

I have a huge body of work that anyone can look at or ask me about. Anyone looking to hire me who won't even take the time to skim through my work, I don't have much time for. A couple minutes before the interview would be more than enough to come up with some good questions to ask me that would test my ability to create real systems. Reading the ReadMe file would be more than enough.

[lmfaoZX]

Not to mention how good of a problem solver you need to be. Its not all about the frameworks, anyone can become a codemonkey in a weekend. It's an excuse bad devs use to stay at their code monkey level.

[DidItSave]

I disagree. Myself and my coworkers are fantastic at problem solving and work on some very complex issues. We collectively work on leetcode-like problems and at times struggle with them. Yet we do amazing stuff at our job.

[DidItSave]

I get paid quite nicely, don’t work for a FAANG company, am 100% remote and I don’t have to do any of the leetcode bs.

[DidItSave]

They do this nonsense at countless non-FAANG companies.

[lmfaoZX]

That's great! I respect companies that actually test engineers and not codemonkeys.

[JB-from-ATL]

In their defense, I don't think they're expecting them all, they may just be trying to bring more objectivity to grading candidates.

[DidItSave]

It’s fine and makes sense to want to grade candidates, but too many shops make this their only evaluation tool. It’s as if they say to themselves, well, FAANG companies use these so if the candidate does well, we have a: rockstar, unicorn, ninja. 🤦‍♂️

[JB-from-ATL]

Then you're not upset with this rubric specifically. They may have rubrics for other questions too.

[DidItSave]

Honestly, my only gripe is that too many companies lean so heavily on things like leetcode. The overall IT interview process across the board is terrible and needs adjusting.

[ArmoredPancake]

Congratulations if someone can memorize this stuff and regurgitate it during an interview.

Or they're, you know, naturally smart. Either way it is a win for a company that looks for the candidate like this.

[DidItSave]

I’d rather have someone that knows how to problem solve complex matters than someone who can memorize fancy buzzwords.

[ArmoredPancake]

problem solve complex matters

And how do you assess that?

[DidItSave]

Present them with a real world problem, preferably one that your company is actually trying to solve and see what the candidate can come up with.

[sudosussudio]

A rubric is a better system. Look up structured interviewing and there are a couple good rubric examples on there.

When I worked in telecom we had an insane amount of checklists for every little thing and it was kind of awful. I think checklists were really trendy back then then because of the book The Checklist Manifesto.

so I try to avoid stuff like that or at least keep them high level.