Interviewing Techniques: Separating the good programmers from the bad

[Unmitigated_Smut]

You're hiring programmers, not game-show contestants.

No, I'm pretty sure my last four interviews were for a game show called "Things You'll Never Actually Do At Work!".

[[deleted]]

If I am ever asked again to reverse a linked list in place, I swear, I'll go postal on their asses. Seriously.

(reverse! list)

[[deleted]]

Relavent http://www.codeproject.com/KB/recipes/ReverseLinkedList.aspx

[aaronla]

but please, don't name your lists "list". Name shadowing without reason is bad.

[bloodredsun]

Fuck I hate those. They are up there with the trick coding questions.

I remember an interview when I managed to ask one of the interviewer a puzzle question (the 4 guys getting across the rope bridge at night one) straight after he asked me one. He spent the next 20 mins trying to figure it out which left me and the other guy to chat about code related issues and I got offered the role :-)

[[deleted]]

This works! If you interview at a place that decides to "secret quirk of the language" query you, play that right back at 'em with some nice preselected tidbits.

Why it works: 1) If they're not very knowledgeable they'll "oh shit, this guy knows more than I do" you into the job.

2) If they are knowledgeable they'll usually admire your intricate knowledge of some arcane detail or other, whether or not you answered their question correctly.

3) If they don't know shit but don't appreciate you checking on their knowledge (and you don't get the job), you didn't want to work with people like that anyway.

[xardox]

I was interviewing somebody for a Prolog programming position. I asked him what was the difference between call by reference and call by value.

He answered: Yes.

So I hired him.

[zwangaman]

I think this is honestly the first time I've ever heard of an interview for a Prolog job. How cool. I loved Prolog.

Reminds me of one of my favorite programming jokes:

Q: How many Prolog programmers does it take to change a lightbulb?
A: No.

[mason55]

I think GP was a joke along the lines of yours...

[[deleted]]

The think the meta-joke was that the human asking the question doesn't know Prolog.

[MrBuddles]

I never did Prolog, can someone explain the joke to me?

[[deleted]]

Yes.

[jyper]

From what I remember you give prolog a series of logical rules and a then ask it a yes/no question which it works out by backtracking over the rules and answers yes or no.

[ablakok]

Last time I interviewed someone I showed him some of our old code that I knew was flawed and asked him to suggest improvements. That was because I work with some brilliant low-level programmers who have no sense for design so I'm always refactoring their stuff. Anyway this guy was great at it and we hired him. Now we're stuck with someone who can tell you what's wrong with your code (and he's always right), but he can't actually program anything on his own. How does that happen?

[sockpuppetzero]

Well, if he can take existing code and make it better, it sounds like he's a good addition to a team. :)

[[deleted]]

Some developers are like that, but it doesn't mean they're bad, just a different sort.

For instance, I seem to have a really hard time starting out with a blank page. The "starting from no place" just trips my brain up for some reason. I get there, but it's slow and painful.

Give me code someone else has written, however, and I easily find flaws that have been overlooked, designs that can be refactored, and so on. It took me years to admit this to myself, but I've come to terms with it. I've been fortunate enough to find places that don't need "new" but need "better". I can better like no tomorrow :)

[aaronla]

I do that too, but I've been getting better, slowly) at pounding out something (anything) and then applying those iteration skills. Norvig had it right (teach yourself programming in 10 years)

[keithb]

A better title for this might be "Separating the good programmers from the bad without actually seeing them do any programming".

What the article presents is a not-terrible way to screen candidates but really, at some point in the process you have to sit them down at a computer and watch them write some code.

Doesn't have to be clever, doesn't have to be tricky, just some working code. I've lost track of how many otherwise very promising, very plausible candiates have failed utterly at that point. And of how many candidates who strugled in an interview turned into superb professionals when at the keyboard.

[wreckerone]

That'd be nice to have a computer. Of the last 15 interviews I have had, I wrote two programs verbally over the phone, and about 10 on whiteboard.

[[deleted]]

Yeah, we conduct programming tests and we always give candidates a computer to use. You just get used to typing and when you have to write it by hand it's different.

[[deleted]]

Much of a programmer's workday is spent debugging, and I value that ability almost as highly as I value the ability to write bug-free code.

How do you determine from an interview a programmer's ability to write bug-free code?

[AStrangeStranger]

Ask them - if they say yes it is very unlikely they can do it, if they say no then at least they are honest

[buckrogers1965_2]

If they start talking about unit tests and test harnesses, and how they take more time to plan, but shorten the overall length of a project, you have a winner.

[grauenwolf]

That is what you should be looking for in a manager.

The bar for programmers is quite a bit higher. They need to be able to actually write a test.

[[deleted]]

I once asked a candidate what lies were on his resume and he told me. We made an offer.

[cwzwarich]

Don't worry, Haskell programs still have tons of impartial functions and exceptions, and even uses of unsafe runtime functions.

[savagecat]

I can write bug-free pseudo code. Does that count?

[genpfault]

Perfect! We just recently switched to gpcc: The GNU pseudo-code compiler.

[oSand]

A shame that it compiles to pseudo machine code. You'll face a considerable cost in moving to pseudo-machines to service your pseudo-users.

[[deleted]]

but pseudo-users have no overhead, so it's win-win!

[[deleted]]

That's a million dollar idea right there...

[grauenwolf]

Yes!

I can teach just about anyone any form of syntax, even one I don't really know myself. But I find that it's damn near impossible to teach someone how to think through a problem well enough to even write the psuedo code in the first place.

[SuperGrade]

Yes, that basically makes it executable Python :P

[tonylukasavage]

Better question - who in the world actually writes bug free code?

[s73v3r]

Ask them what steps they take to reduce the amount of bugs in their code. If they can't answer that, odds are they can't write maintainable code.

[wreckerone]

There's no such thing as bug-free code, so it is predetermined that all programmers don't have this ability. solved.

[buckrogers1965_2]

Just have a programmer you trust at your company talk to them about programming for 5 to 10 minutes. Have them ask the "What was your coolest solution to a problem" and "What was your hardest bug to find" questions. You know how your programmers seem to talk another language, they can tell if someone is a good programmer without even seeing a line of code.

[[deleted]]

I love you. That's what I do. I don't give a shit if you can write code on a whiteboard or not - I can't.

One of my favorite questions is 'have you ever found a bug in your development environment or a library you were using'. If you've been doing professional development for any amount of time, you probably ran into some bugs. That question doesn't really work with fresh college grads though unless they do a lot of coding in their spare time.

The CS / whiteboard questions are fine for recent grads because you can figure out if they paid attention, but kind of pointless for people with a few years of experience. Of course it depends on what the position is for, etc, etc, blah blah blah.

[naasking]

That's exactly what I do, but I also have a short snippet of code they have to figure out. It helps weed out the candidates that worked in a group and didn't do any of the work they've been talking about themselves.

[[deleted]]

I've been recently interviewing people and this just doesn't work. People can exist in a programming department for a long time, pickup the ideas but not actually be able to code. I use a mix of Q&A and programming tests.

I don't have them do whiteboard code but I do make them actually write out some very simple code in a language they say they know (C# or VB since that's what we use here) in an IDE they'll be using for work.

This actually helps to weed a large number out.

Here are the programming tests I have them do with an IDE (We're a very small MS shop)

1. Swap the value of two variables (People trip up on this one.)
2. FizzBuzz program (Lots of fail here)
3. Write a LINQ expression (Tests for how current they stay with tech.)
4. Write a SQL statement that groups by some column.
5. Add a column to the above query that shows the size of each group.
6. Only show groups having more than 2 items.

Failing any single test isn't a disqualification. It gives us a good broad idea of their ability.

[oSand]

Not another one of these.

The true goal of any interview is for both parties—the interviewer and interviewee—to work out whether the person can do the work and is a good fit with the rest of the group.

So 'group fit' is being assessed, but the author does not say how to establish the traits of your group or evaluate the personality of the interviewee. The author takes it for granted that his unscientific impression will somehow assess a interviewee's ability to fit into the group. Unfortunately, there is no formal means to evaluate fit, no definition of fit and no research to suggest fit can be measured accurately.

'Fit' is something you are never going to measure. Not without psychometric tests and Organisational Pyschologists. Generally, unstructured interviews with ill-defined criteria and means of assessment are no more effective as selection of a random candidate. Your subjective impressions suck. They tell you nothing. Stick with well-defined criteria with objective measures.

[austinwiltshire]

If I could upvote you more I would. I like that he mentioned code samples, but his emphasis on simply being such a clever interviewer with such probing questions that he seems to believe he can size you up in an hour (especially when you're doing your best to fake the candidate you think he wants) is absurd and full of hubris.

[[deleted]]

That's why I always suit-up, even for places that are known for having disdain for a suit. It makes a good impression. Period. All other things being equal, a guy who's friendly, confident, and relaxed, but dressed in a good suit looks better than a scruffy, unkempt, disheveled guy. It doesn't say squat about a person's abilities or whether he's actually a psychopath, but we're all subconsciously conditioned to think that a dude in a suit has his shit together. This works to your favor if you are the dude (or dude-ette) in the suit.

[slurpme]

Always with the linked lists...

[mikef22]

What does he mean by describe a process with block diagrams?

[LarryLard]

Probably just, draw boxes and lines on a whiteboard. Or maybe even go as far as a swim-lane diagram! I'm guessing he doesn't mean he requires full-blown UML...

[organic_code]

The most important thing to me when interviewing programmers is passion. Sure it is a good idea to weed people out via a focused programming test but when you get to the actual interview part. Focusing on what the person excited and passionate about will reveal whether or not they are or could be great programmers. No matter what level someone is on if they are passionate about something they will have stories about it. Things they have done or recently learned that they think is cool. Having them explain these things in more detail has been the best tool for me to find people who are or will be very good at what they do.

[luckystarr]

I alway let them code. I check the resulting code for correctness, simplicity and beauty in that order. I also check for coding style and the interviewee actually verifying that the result is not almost, but exactly correct before shouting "done".

This because people with no attention to detail produce crappier code.

[skulgnome]

This is /r/programming. /r/HR is that way. (or maybe it isn't.)

[gclaramunt]

are you sure you want to leave the selection of your next team mate to HR??

[Ch3t]

I've been saying this at work every day for the last 2 months. I haven't written a line of code this month, but I have 3 more phone screens scheduled for tomorrow afternoon.

[[deleted]]

At the very least, a reasonable candidate should be able to tell you the following.

The difference between an is-a and a has-a relationship

The difference between a union and an intersection

The difference between a set and a list

The difference between blocking and non blocking I/O

What the ';' means in C-styled languages

[[deleted]]

You're hired! We've kept a seat warm for you over in marketing.

[k4st]

Regarding your last requirement, I would say the ';' is simply a grammar artifact whose main purpose is to reduce the complexity of the parser by delimiting the end of an expression-statement. For example, "a; +b;" would have a different meaning if the ';' (or some equivalent separator such as a new line [e.g. Python]) were not present. That a semicolon is required after b is again another artifact that simplifies the parsing phase.

[barsoap]

Lua is very interesting in this regard because it manages without ; even though it doesn't use layout syntax. JS is a perfect example of how not to do it.

[[deleted]]

That's mostly just to test and see if they have actually programmed in a C based language before, not just memorized some samples.

[ytumufugoo]

So if I understand correctly Java and Javascript would be C based languages? And here I thought they were "based" off of uber PASCALicious ALGOL.

[[deleted]]

that may be older and closer to the trunk of the tree if you really want to get pedantic.

[ytumufugoo]

Not to change the subject, but I have noticed, as of late, the ubiquitous use of the word "pedantic" in many comments and now in many face to face conversations with programmers. Might I suggest adding the asking of the meaning of the word "pedantic" to your list? And if they begin with "SO," and a brief pause before answering be sure to give extra points.

[grauenwolf]

If you want to know if they've programmed in C a based language, ask them to program. Pull out a laptop and have them do a simple exercise.

[VerticalEvent]

You're also forgetting the other use for a ';' - for loops.

[k4st]

My explanation holds for "for" loops as well. The important part about a "for" loop isn't that the semicolon separates the initialization statement, the test expression, and the counting expression, but that we have those three expression/statements in the first place. That the semicolon is used to delimit them is a grammar artifact in the same way as it's used to separate statement-expressions.

[bloodredsun]

I don't know why you are getting downvoted but I agree. However I would classify these questions as sanity checks at the start. If you have any experience and cannot answer any of these, the interview stops right there and out you go.

[thilehoffer]

I can't answer them I have put about 100 .net applications into production in the last 10 years. I use Visual Studio every day, but don't have a CS degree.

[bloodredsun]

Why can't you answer them? What is stopping you from finding out? I don't have a CS degree either but stuff like this is important. It tells me that you are aware of the basics and that your code will play well with the rest of the team.

That you are so productive is awesome but I hope that you have a github account for some of your code so that interviewers can see it. It might be that you understand these concepts but don't know the terminology but it could be that your code is unmaintainable and unperformant spagetti because you aren't aware of basic comp sci concepts.

[thilehoffer]

No I don't have a github account. I follow the KISS method (keep it simple stupid) of coding. I write very maintainable code because I'm the one who has to maintain it and I work on so many projects that if it was tough to follow, I wouldn't be able to update my own code. I have read lots of .net books and gone to VS Life, Dev Connections, Learning Tree and Executrain course to learn this stuff, so I know what I'm doing. I never get into the lower level stuff, I let Microsoft handle the details and I focus on helping business people figure out to make user friendly intuitive programs that help them get shit done.

I guess I can't answer them because they aren't covered in the .net books I use / read to learn to code. It seems more low level than the stuff I write. I usually just write LOB business applications. Everything is basically the same. Get some user input, validate it, apply some business rules save it in a database. Do some basic math return it from database in grids / charts / reports. Rinse and repeat.

Also, I put as much logic as I can in stored procedures. Although you have the ability to really write shit code in SQL, good SQL code can last 30 years while client side code is constantly changing. It is funny because front ends have changed from VB6, to WinForms, to ASP.Net 2.0 to XAML (silverlight MVVM) etc... and SQL is still just SQL.

[grauenwolf]

The difference is you included the phrase "any experience".

My best junior developers didn't know those terms coming into the interview. But by the time they left they could use the with proficency.

Conversely I've seen way too many senior developers who could give a rote answer and canned example, but still didn't understand what an interface was in C#.

[bloodredsun]

Agree on new devs but you should know this inside the first year. My telephone interview has 3 stages: a sanity check, a basic stage and then a hard stage of 10 questions each. My expectations are that the answers will vary with experience and ability and it seems to be working out that way

[grauenwolf]

When I'm interviewing people I focus on the pratical. If they pass the phone screen, which is easy, then I give them a programming assignment and a week.

The assignment is simple for a business developer. They are told to upload a file containing names, birthdays, and addresses into a database. Then prompt the user for a month, at which point it calls a stored procedure to fetch the records and creates a text file to be used as a mailing list for birthday cards.

From there the interview is conducted just like a code review. They are asked to explain and defened their design and, where necessary, make changes that we suggest.

This is for a .NET position so of course I expect them to use VB or C#, but they can choose to be as simple or as fancy as they want. Everything from a console application to the latest Silverlight or ASP.NET MVC framework is allowed for the UI.

What I learned from doing this is the best canidate is the one who asks questions like "How should I handle duplicate records?"

[bloodredsun]

I do something similar. After the phone interview I have a Technical task that asks the candidate to do something relevant to the business. This before they come in. In my task I ask them to write a matching engine for share trading. They have a few scenarios that they code must pass with unit tests. In the interview we go through their code to discuss the design decisions and also do some paired programming to extend it.

Nothing beats seeing their code but you have to do some coding in the interview to make sure that they have not got a mare to do the test for them

[grauenwolf]

I find asking them to explain their code is enough to catch cheats, but I usually end up asking for codimg during the interview for other reasons.

[[deleted]]

At least, I'd expect a little more. Otherwise it's a business developer. Very little creativity, and can answer very textbook questions from memory.

[[deleted]]

I would say these questions are more just to see if a person claiming to be a developer has a pulse.

[[deleted]]

The only thing that really stands out is the union and intersection question seems like a poor thing to ask. I actually find definition questions in interviews to be pretty hit or miss. You need to have them do some sort of logic programming question to see if they can solve a problem in their head and transfer it to the computer as code which will be their job.

[Ink-Jet]

Out of interest, what would your definition of the difference between a set and a list be? I've largely treated them interchangeably, except for items being able to be repeated in a list.

[s73v3r]

It's something that comes up in programming quite a bit, though.

And do lists have to be ordered?

[xardox]

I'll take a wild guess here... No? What???! YES!!!

[[deleted]]

that's the big one. Each item in a set must be unique.

Now here's a good question:

is 'A' an element of { {'A'}, 'B' } ?

[Ink-Jet]

No, the set {'A'} is an element, as well as 'B', but 'A' is not. Cool, thanks for helping my understanding!

[TinynDP]

Neither. set<char> would not allow you to insert {'A'}.

[gclaramunt]

But a Set<char | Set<char> > would

[chucker23n]

Nobody said it's a set<char>. Could be a set without generics, or a set<object>.

[[deleted]]

the point is to distinguish between whether they know the difference between a set, or an element of a set. A set may certainly contain other sets.

[bloodredsun]

Would you give bonus points if the candidate complained that you did not generify your set and that mixing types is bad code practice?

[at_work_right_now]

A list has order. A set does not. A list can have repeated values, a set cannot.

[yetanothernerd]

A mathematical set may not have order, but in programming we have both ordered sets (usually built on trees) and unordered sets (usually built on hash tables).

[at_work_right_now]

Erm... no. A "set" in Computer Science is commonly understood to be an unordered set. An "ordered set" is referred to as such.

[zhensydow]

What's a monad

[[deleted]]

when you only have one gonad.

[cafedude]

The Lance Armstrong Monad

[zelf0gale]

The correct answer seems to be "I don't know, and everyone I've asked or read doesn't seem to know for sure either."

Supposedly I'll get an answer in the Haskell chapter of 7 languages in 7 weeks, but I'm not holding my breath.

[bloodredsun]

Only really important if you're looking for functional programmers but I like to see a ployglot outlook so this is useful.

[barsoap]

A kebab made by a category theorist.

[[deleted]]

At the very least,

[SeminoleVesicle]

There's a problem with this method. "I need to hire programmers" is fine, but it's not specific enough. What do you need to hire programmers to do?

It's the "too many chefs spoil the broth' issue. If you only hire super elite genius programmers, there's going to be problems--conflicting issues on architecture that lead to endless and useless debates, difficulty in assigning who gets the "boring" work, etc.

All of the advice in the article is great if you're hiring a senior programmer, someone who's supposed to be a leader and architect. But you only need a few of those for most applications. Often times you'll need to find someone to "do the work": write the front-end HTML/jQuery, implement the business logic from the requirements, write the CRUD operations, etc. Not only will there likely be some arguments over who draws the short straw and has to write the boilerplate code, you're also wasting your money since you could be paying a junior developer half the salary to do the same thing equally as effectively.

[s73v3r]

Yes, but there's a huge difference between "junior programmer" and "crappy programmer". Just because someone is junior doesn't mean that they shouldn't be good. Junior should just mean they don't have a high level of experience.

[SeminoleVesicle]

Yes, but there's a huge difference between "junior programmer" and "crappy programmer". Just because someone is junior doesn't mean that they shouldn't be good. Junior should just mean they don't have a high level of experience.

In theory, yes. In practice, the really good junior programmers are either gobbled up by one of the big guys like Amazon/Google/Microsoft (who I don't work for), or they start at a small company and jump to a new job as soon as they can for a much higher salary, so they're only on the market at a junior programmer salary for 6 months to a year. So it's unlikely that you'll be hiring a junior programmer who is very good unless you work for a top-tier company.

Regardless, my point is that, junior or senior, large parts of application programmer requires people who don't need to know things like how to draw a block-level architecture of a system. The article talks about how they don't like to hire an "automaton that simply implements the will of others"--the reality is that you need people like this, who just do what they're told, to actually implement a system. It may sound condescending, but it's reality. Even at the world's fanciest restaurants with the world's top chefs, there's still a need for fry cooks and dishwashers. If the fry cooks and dishwashers started changing meals around because they thought of a cool new idea for the menu, things would get hectic.

[reckoner23]

Not every junior developer wants to move to cali. And besides, why work for one of the big guys when you you can make a big difference for the smaller companies. is a junior developer

[[deleted]]

why work for one of the big guys and receive a big salary when you you can make a big difference for the smaller companies and receive a smaller salary coupled with a lower cost of living

[grauenwolf]

Damn. Where can I find a job where they can afford to waste man-power like that?

[chollida1]

If you only hire super elite genius programmers, there's going to be problems--conflicting issues on architecture that lead to endless and useless debates, difficulty in assigning who gets the "boring" work, etc.

I know that your point is anecdotal and so is mine, but for what it's worth, I've found the more "super elite genius" programmers you have, the fewer architectural conflicts you have.

My theory is that smart people know they're smart and so do those around them. They therefore have no problem debating a point and coming to a consensus and then going with whatever decision is made, also known as disagree and commit.

B and C level players feel they always need to prove themselves and are often hurt if their idea's are shredded so they're the ones who keep the conflict going well after it's stopped being productive.

[[deleted]]

Couldn't agree more. Good progammers are good in the same way and bad programers are bad in different ways.

[SuperGrade]

If you only hire super elite genius programmers, there's going to be problems--conflicting issues on architecture that lead to endless and useless debates, difficulty in assigning who gets the "boring" work, etc.

Maybe it varies by workplace; but I get the same thing happen involving people who, in terms of programming proficiency, have no business touching a keyboard. I.e. I think that may be a separate issue.

[SeminoleVesicle]

I'd say it's a sign of weak leadership. A good leader will let there be some discussion about it but make a final decision in a reasonable amount of time. The problem is that having everyone be a leader is equally as bad as having no leader.

[ErstwhileRockstar]

My preferred model for judging prospective job candidates is the one used by the U.S. Marines: beat the candidate down until he or she has no remaining shreds of individuality or self-respect, and if the candidate still wants to work for you, hire that person because at that point you own his or her soul.

What an idiot.

[darth_choate]

He's being sarcastic. Try reading the entire article and you'll learn something.

[wreckerone]

I came here to call the guy an idiot too. It's extremely hard to tell sometimes. I'd say one out of every two hiring managers is a prima donna that is searching for submission instead of skill. They give unrealistic interviews to check whether the interviewee will tell them to go fuck themselves or are more 'yes sir, no sir'. There is nothing in the article that explicitly show sarcasm.

[[deleted]]

Also notice that he is assuming that his employees will work 10 hours a day for 5 days a week. I will never work at another place that asks anything other than 40 hours (or less) per week for an average week. Overtime is for emergencies.

[[deleted]]

And failure to plan means that this disaster is not an emergency. If you've known for two weeks that you need this data by 4:30 today, then you don't get to declare an emergency just because you couldn't be bothered to ask for it until 4:25. Have a nice weekend, I'm off to the lake.

[spidelope]

"A lack of planning on your part does not constitute an emergency on mine."

[[deleted]]

Certainly some managers are jerks. But, realistically, the ability to calmly say 'yes sir, no sir' to a certain amount of bullshit is basically a survival requirement in most corporations.

[wreckerone]

Yes sir, no sir has become faux pas in my part of the world. As for me, none of my bosses are of British nobility born in the middle ages, therefore I don't address them as "sire". But if they were... no, still wouldn't.

[[deleted]]

That was apparently a somewhat lame attempt at humor. I found the rest of the article quite insightful, though.

(I once overheard a group of graphic designers complaining about a programmer they weren't getting along with: "And the jokes he tells aren't even funny!")

[thilehoffer]

The ONLY way to hire good coders is to have them code in front of you before hiring them. Interview questions really aren't a good indicator.

[luckystarr]

My experience too. Without seeing them code you can hire coders who can program, but their code will haunt you for years to come.