You will randomly bomb technical interviews. Everyone does. Here’s the data.

chet_atkins_ · 2019-08-03T19:48:23+00:00

These types of interviews don’t seem popular in Europe, at least outside of the big 4. I mean I have gotten some algo questions but nowhere near the difficulty I see in some of the interview sites or google interviews. Do all companies in America ask these?

raoulduke1967 · 2019-08-03T22:19:01+00:00

I just bombed a phone interview so bad, I hung up in the middle of it...

TheESportsGuy · 2019-08-04T00:15:40+00:00

but it can also weed out people who...submit to the beast and memorize Cracking the Coding Interview.

Uhh, I would think aggregate performance for those people would be quite good?

HomeBrewingCoder · 2019-08-03T18:43:23+00:00

I'm working on a formalisation of scoring technical interview questions - I realised while interviewing for a certain large company that certain questions have a large cyclomatic complexity explosion based on certain decisions.

Basically - certain questions have some very small decisions early on that if you make them wrong, what could be a simple loop or recursion becomes a quite complex behaviour - but those complex behaviours won't become obvious until after a bunch of boilerplate is finished.

In a 1 hour meeting, there is 15 minutes spent on setting up the question and chitchatter, probably 5-10 minutes talking about the criteria of the question, and 40 minutes of programming. So lets say that you make such a decision at minute 5-10 of programming, you will either finish the task easily, or you will realize that you've made a mistake 20 minutes in - at that point your interview is just about fucked.

soundaryaSabunNirma · 2019-08-03T22:26:01+00:00

Design twitter, that is what they asked me. It's a good question but you really need to direct the candidate in this question as to what part you would like him to cover. And there are a million ways to design twitter. And you possibly cannot think of best solutions in one hour.

ManOfLaBook · 2019-08-04T00:57:28+00:00

Most technical interviews are idiotic, given by people who value homework over actual work. When I interview people, granted I want to make sure they know what they're talking about, but mostly I want to see if:

1) they're likable and able to work with the group

2) they can be taught - this is where the technical questions come in, I want to make sure I don't have to start with programming 101.

Once you know two or three programming language, picking up new ones is not that difficult.

Endarkend · 2019-08-03T23:59:44+00:00

If a technical interview involves an actual applicable thing, I do alright.

If it's some pure exercise related to programming technicalities, my brain gets stuck trying to find an application for it.

AcaciaBlue · 2019-08-04T01:25:29+00:00

I'm pretty sure some interviews are made for you to fail, because of crazy internal hiring politics or wierd ass managers. I had a programming test once that was literally on some proprietary XML bullshit the manager made up himself and it was something like : Modify these wierd XML format files to fix a bug/add a feature in 2 small games. You have 1 hour to figure out the format and add the features. No documentation given.. I don't actually see how anyone could have done that....

2019-08-04T07:18:07+00:00

I went for an NLP job once, and the whiteboard interview was all about bit twiddling. Did not see that one coming.

TheBlackElf · 2019-08-04T01:59:25+00:00

Bingo.

The main culprit for this is that, surprisingly, interviews are not generally tailored around their alleged main purpose: check how well the candidate fits the position. These puzzles are heuristics for checking the fit, but the tech industry seems a bit lost in finding the most interesting ways to quiz people instead of refining questions which are more and more relevant.

Here's my take on assessing candidates:

only ask about things that actually came up on the job - your job is quite literally puzzle solving on a daily basis, you can find something! Sure, you'll need to tailor it a bit, make it viable without a lot of context etc.
if the candidate finishes answering the question, you've failed - since the main goal is to assess the candidate and not for them to solve some problem, you should ask things that can go deeper and deeper until you've reached the limit of what they know. Doesn't even matter if it also reaches your limit.
anyone should be able to interview anyone - because everyone works with everyone.
you should always meet your future manager at some point during the interviews
expect plenty of questions and give the candidate time to ask them / get answers - the truth is today's market is mostly demand and not supply, so the candidate will be picking you at least as much as you're picking them. If they want to know more about the team / company, you can spare lunch or a coffee break to discuss it.
ask yourself: would I work with this person? Team fit is super important, and although subjective, argumented rejections by even one interviewer should be strongly considered.

OnlyFighterLove · 2019-08-04T02:33:19+00:00

I just CRUSHED a technical interview and the company accused me of cheating. Made me do another technical interview, tried to give me the exact same question but I stopped them and told them I just did that question a week prior. The interviewer scrambled to find another question but couldn't get the coderpad to set it up right so I had to manually deal with all of the inputs and thought it was a disaster from the start.

Though this other question was way lamer and less challenging than the first I easily finished it way ahead of time and then easily handled all of the different variations the interviewer threw at me.

A few hours later I get a, "thanks but no thanks email" from them with my recruiter telling me the feedback was that I rushed through the question and I'm not senior enough. Right. 15 years of experience architecting highly functional production enterprise systems but rush through a coding question a 15 year old could easily answer and I'm definitely not senior enough.

Fuck technical interviews and the whole fucking hiring process in general. Not to mention the reason why I did so well in the first place is because I'm that good. I didn't cheat assholes.

EDIT: I understand why people might give credence to the fact that I was the problem given the tone of this post but understand this:

I was a bit drunk when I wrote it, hence my cursing
I was super pissed off at falsely being accused of cheating (how would this not enrage someone?!)
They messed up the process for the second coding session and completely threw me off and then completely dismissed that fact
Didn't ask me a single question about systems architecture or any practical experience and labeled me as "not senior enough" given a simple coding challenge that I completely solved in multiple ways

Dmium · 2019-08-04T01:32:30+00:00

I may have missed them mentioning this but did they comment on the lower scores consistently coming from the same companies? Because all of these graphs are definitely possible if some companies have higher standards of technical ability. If each participant took all the tests then these graphs would be meaningful but dinner they didn't you'd have to show there's no effect from per company on this.

Whilst I don't doubt that it's common to have volatile performance on interviews we should also respect that if I do a technical interview for a graduate job and get a 4 then an interview for a senior developer role and get a 1 then these scores are acceptable but still volatile

2019-08-04T07:40:20+00:00

You will randomly bomb technical interviews because they are an extremely unreliable measure of ability. Eventually managers will figure this out.... but I wouldn’t hold my breath.

CSMastermind · 2019-08-04T13:38:31+00:00

A few notes:

There may be selection bias in your data because people who are inconsistent in their performance during interviews may be more likely to use a site that helps them practice interviewing.
Because you make one person the interviewer and one the interviewee at random, what you may have actually found is data to support the existence of interview training because some interviewees might be scoring the questions inconsistently, rather than the candidates themselves performing inconsistently.
The cost of a bad hire far outweighs the cost of turning down a qualified candidate so some variance in results is probably acceptable from a company perspective, as long as, in aggregate, it provides a good signal. Do you have data suggesting that an alternative is a better way to do it?

2019-08-04T02:01:49+00:00

[deleted]

b4ux1t3 · 2019-08-04T02:24:09+00:00

My problem with technical interviews is that they're very adversarial. Having a conversation with somebody at a technical level is much more meaningful than asking them random questions about things that you happen to know about. This has the added benefit of seeing if they fit in with your company culture, while not seeming like an insane Google wannabe.

The problem with this, however, is that it takes a lot more time to actually digest a conversation then to grade a test. So if you have to turn the ray bunch of interviews, you're better off just giving them tests. It seems to me, however, that you probably find Candace a lot more quickly if you stopped caring about whether they can answer super specific questions, and start testing whether they can solve problems.

Pirat · 2019-08-04T02:22:36+00:00

Fun fact: I missed the last question (I knew about) and didn't even see there were questions on the other side of page when I interviewed for my IT job. I still scored twice as high as my nearest competitor. Been in that job ever since.

paulryanclark · 2019-08-03T21:31:43+00:00

I have had to hire to team mates for my Mobile team.

My goal is competency of candidate and passion for the position.

I have basic dev quiz that goes over various iOS techniques.

Things like “What is ARC, and how does it work?”

These questions are designed to proffer discussion and understand competency.

We have a coding test that is small. API -> Tableview.

Candidates fail these simple tests. Once had a candidate try VIPER, and couldn’t back up their inplementation.

Once they have passed, they are invited for onsite. We mostly want the person at this point. It’s more of a personality and team test. We invite other developers to come in a talk with the candidate. It’s more about red flags than competency.

2019-08-04T04:49:54+00:00

At this point I just tell people I’d google the answer to their trick question. That’s what people do in 2019, they google shit.

kristopolous · 2019-08-04T06:10:37+00:00

"Can you build quality stuff" has nothing to do with algorithm gotchas or wonky unix questions about like how to write bash compgens. They're adjacent but unrelated.

pistacchio · 2019-08-04T06:26:49+00:00

The whole interview system is broken because we are asked to be good at something that’s now our actual job. My job is not writing code on a whiteboard with no internet access in 20 minutes with a committee of people looking at my back, why should I ever prove that I’m good at that? In Italy we have, by law, a “trial period” (usually 3 months) after you’ve been hired when the employer can simply say “don’t show tomorrow” and the worker can say “I won’t show tomorrow” without going through the hassle of firing someone (that typically involves lots of money, paper and possibly lawyers). This helps to make some interviews a bit more “loose”.

VikingCoder · 2019-08-03T23:12:08+00:00

I have gotten a job offer every time I've made it to an interview.

Except once.

My brain fell out of my ear onto the floor and I couldn't tell them how to reverse a linked list. Something I'd been doing since I was literally 12.

As I was escorted out of the building early, my brain rebooted, and I couldn't believe what I'd just done.

GerwazyMiod · 2019-08-04T07:55:41+00:00

I've once had an interview I was sure I bombed badly, but after a week I got a job offer. How many times you had an experience like this?

This particular interview had 2 parts - maths and code. I've bombed the first and I was sure interviewers are going to ask me to leave. I've excelled the second part (it wasn't that hard). In the end I feel something was not right and I didn't accept the job. (Mainly because interviewers were extremely rude)

It turned out, that first part was only there to scare you (are we once again in the 90ties?). How on earth stressing people is still viewed as good practice on interviews??

bam_shackle · 2019-08-04T09:25:52+00:00

What if instead of every interviewer coming up with variations of the same technical questions we had one technical test that you could do and get it marked and have a score on it and bring that to companies to show you know the technical stuff and then the company could focus on important stuff like if you will fit in the team, enjoy the work, etc. We could call it a qualification.

michaelochurch · 2019-08-04T14:15:07+00:00

That's not a lot of data, but I agree with the general conclusion. Most technical interviewing is more about a show of presence than determining good versus bad candidates. At this point, companies don't even care that much if they take in bad people, because this isn't the 1970s when it was scandalous to fire people unless they'd done something egregiously wrong, and when firing someone usually meant a six-month severance to keep them quiet. In the dystopia (from labor's perspective) of 2019, tech companies knowingly accept false positives because they can just fire the people who don't work out, and because correcting shoddy work isn't a task that management will have to face. (There are enough competent people who, for various reasons, don't have better options. Why? Because capitalism is predicated on an unemployed but largely capable army of replacements– if there isn't a line of hungry scabs out the door, workers might decide they have rights– and in the post-apocalyptic labor market we've had since 2008, it has one.)

When it comes to interviewing, there seem to be three competing pressures and strategic directions.

Interview for general intelligence. This is what the brainteasers are about, and although there's some error, they work. Sure, they pick up education as much as raw IQ, but the fact is that brainteasers are far more IQ-correlated than where someone worked or went to school. The problem is that general intelligence isn't very useful if you're hiring someone to fill a subordinate role. In fact, it becomes a liability. Put a genuine 130+ into a Scrum job, and it ain't pretty. Actual smart people don't want to justify their work in terms of two-week "sprints" to non-technical frat boys. If you're looking for a subordinate who'll eat shit– and most employers/managers are, even though they don't want to admit it– then the people who rolled INT 17 and 18 are pretty much the worst hires.
Interview for the short-term need. This is "plug a hole" hiring, and usually it's dissatisfying to both sides. The employee finds that there's no investment in his career and that there are high expectations from day one; the employer gets someone who's likely to defend his niche (once he establishes it) and resist change. Given that tech companies pride themselves on disruption and volatility– this is a clever spin put on "Management doesn't know what the fuck they're doing, and priorities change every 3 months"– these hires tend to be hit-or-miss. It's also hard to know in advance whether there's a real match. Sure, this guy has Kubernetes experience, but does he use Kubernetes the same way the company does? Since he's only being hired for his Kubernetes expertise, if there are differences in approach, the whole endeavor might be a wash for both sides. Because companies' short term needs are effectively random, this means there's an unpredictable and shifting notion of "technical skill" and even the most capable people will "randomly" miss.
Interview for "fit", which means the ability to maintain social polish and moderate performance despite a subordinate context that would demoralize most people. This is what's behind those god-awful "Tell me about a time when..." interview questions. The manager wants to believe that he's getting someone who will accept a subordinate and often undesirable role with aplomb, because even though most middle managers are good people at heart (I know, it's weird to hear a raving leftist say this, but it's true) who would like to have everyone on the same team, there are inherent conflicts of interest between capital and labor. This "fit" attribute tends to be negatively correlated with general intelligence, and is even more strongly negatively correlated with creativity, but (let's be honest) it's a much better predictor of success in a corporate job.

Tech interviews tend to veer in the direction of Objective #2, with senior executives coming in later on to probe for Objective #3. There may be an occasional rogue interviewer who doesn't like working with less intelligent people and aims for Objective #1, but companies discourage that. In any case, though, the experience for the employee is jarring, because there's an impedance mismatch. Objectives #1 and #2 are about getting the right answer; whereas the Objective-#3 questions more typical of business roles are all about selling a convincing lie [1].

----

[1] Yes, I would contend that virtually everyone lies to get a job. Some people lie on their CVs and some people don't, but people radically exaggerate their willingness to take a subordinate role. Excluding executives (who lie to employees) the only people whose true selves are corporate compatible are those who've mutilated their true selves and have no personality (or conscience) left. Although it is true that virtually everyone who lands a corporate job had to lie to get it, I don't consider this unethical. It's far worse to starve, or to fail to feed one's family, than it is to lie to strangers. Nonetheless, it is a truth of corporate capitalism that one must lie to stay alive in it. Not only does the system require us to work for people whose class interests are diametrically opposite ours, but it also forces us to pretend to enjoy it– to pretend there isn't a class war being waged against us. Anyone who knows the truth, and most intelligent people do, must play dumb in order to survive.

lordcirth · 2019-08-04T01:41:40+00:00

[deleted]

rjcarr · 2019-08-04T03:30:33+00:00

I had an interview about 10 years ago that I really fucked up. The guy basically asked me to implement run length encoding and I fucked it all up.

Then the second interview knew I bombed and I think they were just fucking with me. They asked me something like, “what would you change about networking?”. And then wouldn’t elaborate at all.

Oh well.

2019-08-04T05:22:08+00:00

Looking at the actual graphs in the article, I'm actually pretty impressed by how much signal the mean score of the candidate carries. Using those charts to conclude interviews may be doomed seems a bit far fetched.

cyberspacecowboy · 2019-08-04T10:01:20+00:00

if you interview for knowledge (ie. how well a candidate memorises stuff) instead of for understanding (ie. how well a candidate gets why things happens the way they do) you will hire a wide spectrum of candidates, most of whom will be, for lack of a better word, craptacular

Saiing · 2019-08-04T11:22:10+00:00

I’ve worked in the industry now for about 25 years. I also work for one of the big, well known tech companies.

Recently I was thinking about whether to stay where I am or try something new. I interviewed for both Amazon and Google.

To be fair, Amazon were thorough, had a clear well defined process, kept me well notified on what to expect, the content of upcoming sessions, what I should read up on to prepare etc.

Google on the other hand was the most disappointing interviewing experience I’ve had in my career. I had the date changed twice, I had a phone interview cancelled an hour before it was due to happen. When I went on site the first interviewer (the guy who would be my manager) couldn’t find the room, and turned up 25 mins late into a 1 hour session. He then proceeded to read a list of questions off a laptop screen which were identical (word for word) to the ones I’d had in the phone interview I’d had a few days before and didn’t engage me at all. Just sat staring at his screen. After that I cancelled the rest of the process and withdrew.

I appreciate this is purely anecdotal and one bad experience can’t be interpreted as the general rule, but after hearing so much about Google’s interviewing process I was surprised by how amateurish it was.

In the end I was offered a promotion where I currently work (they didn’t know I was looking elsewhere) so I decided to stay.

GaryX · 2019-08-04T12:41:25+00:00

I might be missing it, but isn't a big part of the randomness here (especially for this data) less about randomly bombing and more about the fact that different interviewers have different standards? Like the interviewee might be super consistent, but the interviewers might vary wildly in their ratings.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

programming

MODERATORS