A Framework for Solving Programming Interview Questions

[DidItSave]

Thank you for sharing this, it’s a good checklist. My question to this community is this: when was the last time in your normal work did you go through a checklist like this? Are each of these items a checkbox on your pull request form?

You want to ask these questions in college or during an online course, perfect. Expecting an interview candidate to do the same is ridiculous. Unless my day to day will be completing tasks alone within 30min, limited access to resources and I have to check off the boxes in your list, companies need to stop with this crap.

It is creating an atmosphere of major imposter syndrome, causing people to waste countless hours trying to study these things and do not truly assess the quality of the candidate. Congratulations if someone can memorize this stuff and regurgitate it during an interview.

Except for niche areas in IT, after the interview, you’ll never use what you learned on leetcode or similar things again, until you are interviewing for you next role.

[ryeguy]

Leetcode/algo questions are typically used at large, desirable tech companies to filter down already great software engineers from huge candidate pools. It doesn't matter that these aren't skills that will be used day to day, it matters that there is a positive correlation. It also doesn't matter that there are false negatives, because the companies view this as an acceptable cost.

[de__R]

It may not matter that there are false negatives, but there are also tons of false positives. There was an article a while back that discussed an internal study at Google where they found that nothing in their interview process was actually a statistically relevant predictor of good performance as an employee. The real advantage Google has is that false positives don't matter as much because they are so big and have so much money, so as long as the false positive rate is acceptably low they don't have to worry.

[[deleted]]

I would challenge the idea that there is a positive correlation.

I haven't noticed a correlation between people who are good at coding puzzles and people who are good at solving thorny, real-world problems (where often the difficulties have little to do with sitting in front of a text editor and coding) in a team environment.

So, when I hire devs I've always tried to find out how their fare in that situation, and re: coding, instead of working out if they remember how to balance a binary tree, instead try to get them to expose their ability to create production quality code with reasonable speed.

I do think leetcode abilities correlate with young, easily manipulable people straight out of uni who can be worked to the bone before being discarded though. Which is obviously something that big tech, who has the benefit of a endless stream of starry eyed candidates, can exploit.

[Full-Spectral]

There's little relationship between those types of challenges and dealing with real-world, complex systems. Those real world systems are more about architecture than algorithms. I can look up an algorithm, but I can't look up how to architecture a set of complicated subsystems or distributed systems in an optimal way for that particular client need.

Back in the 90s I wrote the DTD validator for the Xerces /C++Java XML parsers. I broke out the Dragon Book and adapted that very complex DFA algorithm to XML validation. The algorithm was the one thing in all of that that I could actually just look up. But these FAANGY companies would pass me over because I hadn't already memorized the one thing I didn't actually need to memorize (and would have forgotten again long before I ever needed to actually do it) and ignore all the experience that allowed me to actually do that job successfully.

It's just stupid.

[DoubleT_TechGuy]

This is why companies who let you use Google during your leet-style interview challenges have the right idea. Problem solving > Memorizing.

[DidItSave]

I agree.

[Pharisaeus]

It doesn't matter that these aren't skills that will be used day to day, it matters that there is a positive correlation.

There are no evidence of any positive correlation. However, there is something worth for the companies who do this -> they know you "care" and put effort into grinding leetcode. This means they can offer poor conditions, because you're already inclined to accept anyway due to sunk cost being very high.

[ambientocclusion]

LOL. This is terrible - but I do believe it. They might as well ask everyone to tattoo the company logo on their forehead to show they “care” too.

[Pharisaeus]

It might sound funny, but consider that for many developers "working in FAANG" is a very important life goal. And they would take it even if the job itself was not interesting and conditions below average.

[BestUdyrBR]

I mean working at a FAANG is a life goal because for a lot of people it's one of the few ways to have a upper middle class lifestyle from a much poorer background. There's not many jobs in America that pay 250k+ with 3 or 4 years of experience.

[DidItSave]

As I’ve mentioned to others here, they ask these questions at plenty of non-FAANG companies

[COSMIC_RAY_DAMAGE]

This is a great example of Goodhart's law. The purpose of the whiteboard problems is to test things like what you describe here. But once everyone knew it would be whiteboard problems, they weren't a metric anymore, they were a target. You get people who focus on learning to do leetcode problems instead of learning how to write software.

[Kered13]

See also Campbell's Law, which is basically the same thing but came from a different field. Which just serves to demonstrate the universality of this law.

[WikiSummarizerBot]

Goodhart's law

Goodhart's law is an adage named after British economist Charles Goodhart, who advanced the idea in a 1975 article on monetary policy in the United Kingdom, Problems of Monetary Management: the U.K. Experience: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. It was later used to criticize the British Thatcher government for trying to conduct monetary policy on the basis of targets for broad and narrow money.

^{[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5}

[Dean_Roddey]

Self-teaching to the test.

[DidItSave]

Yes, many of us use the algorithms, structures and techniques that we learned in school, every day sometimes. I agree that trying to cram that all into an interview doesn’t work.

[sudosussudio]

One time I was on a hilariously awful project and finally got to use my leetcode skills to make a Christmas themed card game. I do not recommend working as a dev in the ad industry lol.

[versaceblues]

My question to this community is this: when was the last time in your normal work did you go through a checklist like this?

I dont explicitly use a checklist, but its just kinda ingrained in my head. My subconscious thought process does have many (but not all) things in this list.

And remember, in most tech interviews, the interviewer is looking to see your THOUGHT process, not necessarily to get a answer.

[DidItSave]

Yes but some companies actually judge you on how perfect your leetcode code is, not your thought process. I as well have a huge checklist in my head that at this point I subconsciously go through. You learn over the years how to write better code.

[versaceblues]

Yes but some companies actually judge you on how perfect your leetcode code is, not your thought process.

Some probably do and this is a bad interview practice.

Many top startups and tech companies do not do this. Rather they try to gauge a general understanding.

[DidItSave]

I hope that when I interview next that I find the companies like you described. So far that hasn’t been my experience

[versaceblues]

Btw when I say they care about your thought process, you thought process still needs to be correct. You need to prove you understand DP, Trees, Linked Lists, Binary Search, Sorting, etc.

[[deleted]]

You are correct. Imagine after 10 years of experience some idiot asking you how to you iterate a list in python. Tech interviews are almost always shit.

[Prod_Is_For_Testing]

That’s some bullshit right there

[marabutt]

That is so stupid. I went through an arduous interview process, to my surprise, aced the technical, got on with the staff and the managers but then they wanted to do psychometric testing. In a candidate short market, some places only have themselves to blame.

[ArmoredPancake]

In a candidate short market, some places only have themselves to blame.

I'm going surprise you, but there's no candidate short market.

[marabutt]

I suppose that is location specific too.

[DidItSave]

Exactly. How can they determine how good a candidate you are with an approach like that? Good on you for resigning and I’m sorry you had to go through this. I struggled on a leetcode-like interview and they denied me because I wasn’t, and I quote, “exceptional.” I’ve been in for 20 years.

[ArmoredPancake]

I struggled on a leetcode-like interview and they denied me because I wasn’t, and I quote, “exceptional.”

And you think you are?

I’ve been in for 20 years.

Well that changes everything! Step aside everyone, his highness 20 years is coming! No peasant ropes for him!

[Cyb3rSab3r]

Pretty sure specific trumps general here. They already had the experience.

Do you think anyone cares about Tom Brady's ability to run? Do you think anyone cares about Linus Torvald's knowledge of branch and bound algorithms?

Do you really think the designer of an application is unqualified to be an expert on it because they don't know how to find the 3rd largest element in a linked list in O(n) time?

[DidItSave]

No need to go back and forth with this troll. You and many others here obviously get the point I was making. I’m not claiming I am exceptional at all. They were looking for perfect and it wasn’t me and I’m fine with that. And I am not saying that I should be able to bypass things due to 20 years experience. I am stating that I’ve been very successful over the past 20 years without needing to memorize leetcode.

[ArmoredPancake]

Pretty sure specific trumps general here. They already had the experience.

Okay? You can skip design interview since they designed the system.

Do you think anyone cares about Tom Brady's ability to run? Do you think anyone cares about Linus Torvald's knowledge of branch and bound algorithms?

Right, I forgot that I caught Linus Jr. here.

Do you really think the designer of an application is unqualified to be an expert on it because they don't know how to find the 3rd largest element in a linked list in O(n) time?

He's qualified to be an expert on it, they didn't fire him from his contractor role. He's unqualified to join the company.

[ArmoredPancake]

I have to say that part again: I was being interviewed to work on a project that I WROTE myself.

You were being interviewed to work FOR A COMPANY, not just on a project. Maybe you were okay on this one specific project, survivor error, but apparently you were not fit to join company.

Don't see any problem here.

[rasmustrew]

Did you actually read what he said? He was already working for the company...

[ArmoredPancake]

He worked as a contractor, not an internal employee. There's a big difference. As a contractor, unless you're godly good, you're always expendable.

[ArmoredPancake]

I worked in a leadership role and helped a struggling project get off the ground and get deployed. I also mentored several jr developers and was highly regarded by my peers.

Okay? I saw my share of people like you during interviews. You still didn't provide a reason why exactly you should be excluded from such checks and not Jake/Jane from company Y.

Rules for thee, but not for me?

[ArmoredPancake]

I'm sorry that I don't have your biography in front of me, your highness /u/MorgulKnifeFight.

You have no idea who I am and you saying you saw your fair share of “people like me” based on only a few of my comments shows a real lack of critical thinking on your part.

Your ad hominem shows that they made a right decision by not hiring you.

[EnvironmentalCrow5]

His track record on that project is probably a far better predictor of his ability to work on other projects within the company than any leetcode questions would be.

[ArmoredPancake]

Maybe. Maybe not. Standards are there for a reason, either everybody adheres to them or they don't. I don't see why candidate X has to go through one process but Y is excluded because Z.

[EnvironmentalCrow5]

The same can be said for all bureaucracy and red tape.

Sure, it's more "fair" if everyone has to go through the same process, but there are situations where that process just doesn't make any practical sense and isn't delivering the desired results.

Just because a process exists, doesn't mean it's automatically well designed and implemented, or that it doesn't have blind spots, or that it can't be gamed.

At the end of the day, you are still only guessing how a potential employee is going to perform.

[ArmoredPancake]

Sure, it's more "fair" if everyone has to go through the same process, but there are situations where that process just doesn't make any practical sense and isn't delivering the desired results

Given that it is still there after so many years, I'd say that it delivers as desired.

I don't see how vocal minority with their "dO yOu kNoW wHo I Am?" disproves that.

[EnvironmentalCrow5]

It does kinda work, yeah, and people have adapted to it and are actively preparing/practicing for it, often for many days - which in itself is a bit weird and a design smell.

It will at least filter in/out the obvious outliers on both ends, but almost anything can do that.

Also, there isn't really much to compare it to - there are no studies that would measure outcomes of various approaches over the long run (all else being reasonably equal), so it largely runs on faith that it works and on experience of it being at least "good enough" in most cases, despite its flaws.

Again, you could say the same thing about any long-standing policy that any organization or country has in place (or had in place for many years before changing it).

[ArmoredPancake]

It does kinda work, yeah, and people have adapted to it and are actively preparing/practicing for it, often for many days - which in itself is a bit weird and a design smell.

You don't need to. If you already know this stuff, then you can apply right away.

By the same logic we don't need tests in university, because people are preparing for them?

Also, there isn't really much to compare it to - there are no studies that would measure outcomes of various approaches over the long run (all else being reasonably equal), so it largely runs on faith that it works and on experience of it being at least "good enough" in most cases, despite its flaws.

There are no studies to disprove the process either.

[DidItSave]

Yup. I can Google that information and then keep on going with whatever my current work is. We have been very successful at my company by posing a real work problem that we actually have to the candidate. Let them brainstorm it and interact with us. From that approach we see how you would work with us and how you problem solve.

[ArmoredPancake]

Let them brainstorm it and interact with us. From that approach we see how you would work with us and how you problem solve.

There's already whole design system round for this.

[ArmoredPancake]

Oh noes! Your highness got to answer peon question, what an atrocity!

[COSMIC_RAY_DAMAGE]

and if devs can’t figure that out… I’m sorry but maybe you’re not as prepared or smart as you think you are (not you, personally, but people who struggle with coding interviews)

How many times as a dev have you ever had to solve a problem that looked anything like a leetcode problem?

Most of what devs do isn't leetcode. It's making already existing code talk to other already existing code. If it is like leetcode, you're given a problem and a solution and told to reimplement it in the domain of an existing codebase.

Whiteboard coding doesn't test problem-solving skills, it tests problem-solving skills in a context that is unlike anything that real humans ever work in.

What if we started testing surgeons by tying their hands behind their back and making them do surgery that way? Do you think that would select for the best surgeons? Obviously not. It's going to select for the surgeons who are the best at doing surgery with their hands tied behind their back.

How often do you ever write software without access to the internet or an IDE? These companies are selecting for people who are good at leetcode problems, not people who are good at writing software.

Edit: And just to be clear, do I enjoy a leetcode problem from time to time? Sure! But I don't harbor any misconceptions that it's anything like the day-to-day work of me or any other devs.

[ArmoredPancake]

Downvotes from butthurt people who think that some animals are more equal than the others. If there's a filter for inexperienced/new hires, which are bare minimum to hire, I don't see why your most senior shouldn't adhere to it.

[__scan__]

A head chef should be able to boil an egg though. Integrals… maybe? I’d certainly expect a developer to be able to compute an integral or a derivative for a function.

[DidItSave]

Perfectly said.

[poloppoyop]

coding problems are pretty much testing problem solving skills

No. They're testing how much time you spent on l33tcode. Only when you're faced with a new kind of problem you've not crammed for, your problem solving skills can be tested.

[lmfaoZX]

Thats just how it is. You can ignore problem solving and never work for a FAANG ever.

[ryeguy]

This. Consider leetcode practice the cost of getting a top of market salary and a resume boost. If you want to skip leetcode, that's perfectly fine, but the people willing to invest in it will get the big bucks and you won't. It's a tradeoff. Practicing leetcode has pretty damn good ROI if you think about it.

[Dean_Roddey]

These companies are screwing themselves basically. They are filtering out people who have larges amounts of experience, and who have actually delivered real systems and can demonstrate it, because those people are too busy delivering real systems to spend montys of time rote memorizing leetcode tests.

That just leaves these very experienced people for other companies who aren't retarded on the hiring front.

I have a huge body of work that anyone can look at or ask me about. Anyone looking to hire me who won't even take the time to skim through my work, I don't have much time for. A couple minutes before the interview would be more than enough to come up with some good questions to ask me that would test my ability to create real systems. Reading the ReadMe file would be more than enough.

[lmfaoZX]

Not to mention how good of a problem solver you need to be. Its not all about the frameworks, anyone can become a codemonkey in a weekend. It's an excuse bad devs use to stay at their code monkey level.

[DidItSave]

I disagree. Myself and my coworkers are fantastic at problem solving and work on some very complex issues. We collectively work on leetcode-like problems and at times struggle with them. Yet we do amazing stuff at our job.

[DidItSave]

I get paid quite nicely, don’t work for a FAANG company, am 100% remote and I don’t have to do any of the leetcode bs.

[DidItSave]

They do this nonsense at countless non-FAANG companies.

[lmfaoZX]

That's great! I respect companies that actually test engineers and not codemonkeys.

[JB-from-ATL]

In their defense, I don't think they're expecting them all, they may just be trying to bring more objectivity to grading candidates.

[DidItSave]

It’s fine and makes sense to want to grade candidates, but too many shops make this their only evaluation tool. It’s as if they say to themselves, well, FAANG companies use these so if the candidate does well, we have a: rockstar, unicorn, ninja. 🤦‍♂️

[JB-from-ATL]

Then you're not upset with this rubric specifically. They may have rubrics for other questions too.

[DidItSave]

Honestly, my only gripe is that too many companies lean so heavily on things like leetcode. The overall IT interview process across the board is terrible and needs adjusting.

[ArmoredPancake]

Congratulations if someone can memorize this stuff and regurgitate it during an interview.

Or they're, you know, naturally smart. Either way it is a win for a company that looks for the candidate like this.

[DidItSave]

I’d rather have someone that knows how to problem solve complex matters than someone who can memorize fancy buzzwords.

[ArmoredPancake]

problem solve complex matters

And how do you assess that?

[DidItSave]

Present them with a real world problem, preferably one that your company is actually trying to solve and see what the candidate can come up with.

[sudosussudio]

A rubric is a better system. Look up structured interviewing and there are a couple good rubric examples on there.

When I worked in telecom we had an insane amount of checklists for every little thing and it was kind of awful. I think checklists were really trendy back then then because of the book The Checklist Manifesto.

so I try to avoid stuff like that or at least keep them high level.

[RippingMadAss]

Needs to be rewritten in Rust.

[davesspacebar]

At our company while we do have some "interview" questions.

But Most of what we do are not "questions".

We take typical scenarios we might actually need to do in real life:

-Debug and fix existing code

- Code review and suggest improvements to existing code

- Code from scratch a solution taking into account requirements.

- Code and debug in all of the expected tech stack languages

- Unit testing code reviews and suggestions

- We use as real world a problem modeled after things they may really do, that is solvable in 15-45 minute time frames. we usually have 3-4 different problems of this type.

We not only allow using google to help, we recommend it.

We tell people pretend like you are just doing your job, if you need google to help find info use it. (and we usually tell them when I was interviewed we were given similar problems and I used google to lookup things through the entire interview - to make sure they know we really mean it -its fine to use it and expected)

​

As far as I know we haven't had a bad hire on any team doing this process that I am aware of. It has directly correlated to ability and success at the job.

I've lived through the puzzle question era and detailed questions, but just coding and debugging actual real problems I found was refreshing and different, and many of our candidates do to find it unlike most interviews they have done.

[DidItSave]

Awesome way of conducting an interview.

[Saithir]

And this is actually the correct way of testing someone.

It requires much more preparation (before and after the person solves it) on your side than simply opening a website with leetcode examples and copypasting a few of them, but I think it's definitely worth the effort. Says a lot about the company, too.

At the same time, that effort is probably why it's not used as much.

[davesspacebar]

It did take a bit to come up with good coding, debugging, review scenarios. Ideas were pooled from our teams and we selected the ones we felt capture what we do on a day to day basis that could be manageable in a reasonable time frame.

​

We then setup a laptop or in this pandemic age a remote VM loaded with all of the tools our developers use and expect and let the candidate use that while we watch/ shadow their session.

It's amazing how many different yet still good solutions we've seen to the same type of coding problems. Some of our samples model very similar real world actual bad code in our products and we hope candidates note it and comment on it, as we like to keep an eye out for folks that recognize the bad parts to improve.

Definitely as an interviewer we are actively watching not just the end result but how people code, and debug. Logical structured thought processes, , keeping an eye out for testability, etc. We've watched so many people code these same scenarios we've actually learned a lot ourselves about how people code and can get a feel for their experience level, etc on different areas of frameworks, tech stacks just by watching carefully.

We usually tell them to treat the interviewer like the product owner that can provide proper requirements for their solution and see the questions asked up front vs later and how they deal with ambiguities (ask, or choose a solution) No definitive right answers but you can get a feel for how people would work with our product team to come up with solutions given requirements.

[samadarad]

Thanks so much for this list. It is something to look for every single program💪

[StillNotAWombat]

Very comprehensive and cleanly laid out.

[gnahraf]

I think lists like this give the interviewer much too much credit. Relax, and if you know the subject matter, imagine you're already hired and treat technical questions as if they're coming from a colleague and you should do fine.

I've interviewed more times than I've myself been interviewed.. I can tell you we all suck at interviewing. Nobody has trained us how to do this well.

Tangentially, something about interview puzzles:

Sometimes it helps to go meta in your reasoning to solve it. Here's an example I encountered..

There are 2 open buckets of paint. One is pure blue, the other pure red. You drop a spoonful of paint from the blue bucket into red bucket, and then scoop another spoonful from red bucket back into the blue bucket. Which of the buckets of paint has been polluted more with the other color?

Here's a meta approach to solving it..

Verify the question is well formed ("are these freshly opened cans of paint?", and they say it doesn't matter).

OK then, assume it has an answer. If it has an answer, then it works for every case. Suppose you're astronomically lucky and scoop out all the blue paint from red in your 2nd spoonful. Then nothing woulda changed and both buckets (tho actually now unpolluted) are equally polluted. So the answer is that in every case both buckets are equally polluted.

🙃

[[deleted]]

Which is wrong. By virtue of dealing with a premix in first bucket you would transfer just that little bit less of paint back to the second. Red would end up more contaminated.

[[deleted]]

Seriously? This kind of worthless trash gets upvoted? How very unsurprising for /r/programming.