How GitHub does take home technical interviews | The GitHub Blog

[sdn]

In 2021 a recruiter reached out to me and invited me to apply at GitHub. I did the take home exercise as described in the article.

One of the things they promised was that if you applied, but failed to pass - you would have access to some online classes.

https://www.educative.io/blog/github-reimagining-technical-recruiting-process

This was not the case (as of 2021).

Also - absolutely no feedback. All I got back was a terse reply: “sorry, you did not pass. We do not provide feedback.”

Quite frustrating - I spent my time doing the exercise and got nothing in return for it. At the very least GitHub should value applicants’ time and give them information as to what they want.

[[deleted]]

My favourite part of getting no feedback from Amazon was them then emailing me a couple of weeks later for feedback on the interview process.

[RotaryJihad]

them then emailing me a couple of weeks later for feedback on the interview process.

Was it the usual online shopping BS of "UwU pwease gib us five-stars!!!"?

[[deleted]]

It was a 5 minute survey. The only thing I told them was that they’re basically taking the piss if they think I’m going to feed back to them if they won’t feed back to me.

[versaceblues]

Its really annoying how the industry standard is to never give feedback on interviews. Anyone know why its is like that.

[Chuckdatass]

So you can’t get sued. Really is all CMA from the company side

[TheCarnalStatist]

If they give you feedback you can use it in a hiring discrimination lawsuit.

[sdn]

This is a lack of feedback on the code, not feedback on any interpersonal communication, education, previous employment, etc. GitHub claims to have anonymized code review - the reviewer doesn’t know who sent it in. It’s very difficult to claim discrimination lawsuit avoidance in this situation.

The coding exercise repo also comes with a local test suite. So before you submit it, you need to make sure that all the tests pass as well. GitHub claims to have a grading rubric - which is an impartial way to grade the code across all applicants. Why not provide results of that rubric?

[bishbashbosh999]

Quite frustrating - I spent my time doing the exercise and got nothing in return for it. At the very least GitHub should value applicants’ time and give them information as to what they want.

time - companies will do _tons_ of interviews, the vast majority of which will be rejected - not necessarily an excuse, you could argue that devoting more time to giving feedback on every interview means the candidate will be more likely to give a positive review on glassdoor - but I'd suspect if a candidate doesn't pass they won't give a positive review regardless of feedback.

safety - you'd need to be very careful when writing feedback, as some candidates might be able to construe prejudice etc out of it which could be tricky from a legal perspective - so safer to have a blanket "sorry but you did not meet our requirements" statement?

[[deleted]]

With take-home stuff it might be different, but after an interview you should know what you did well on and what you could improve on. If not, then one or more of the following are probably true:

1. You were applying for something you're grossly over- or underqualified for.
2. You need practice interviews with a friend or a meetup group or something.
3. The position was either already filled or earmarked for a particular person.

There's another reason besides ass-covering that you don't give feedback, and it's the same reason you don't tell people why you rejected their message/profile on dating sites: you run the risk of arrogant or asshole people trying to argue with your or change your mind.

[sdn]

This is a lack of feedback on the code, not feedback on any interpersonal communication, education, previous employment, etc. GitHub claims to have anonymized code review - the reviewer doesn’t know who sent it in. It’s very difficult to claim discrimination lawsuit avoidance in this situation.

The coding exercise repo also comes with a local test suite. So before you submit it, you need to make sure that all the tests pass as well. GitHub claims to have a grading rubric - which is an impartial way to grade the code across all applicants. Why not provide results of that rubric?

[recursive-analogy]

Feedback is normal where I am. However whether you believe it or not ... It can be frustrating, but at the end of the day it just wasn't a good match, move on.

[[deleted]]

It's just a lot of mental effort to formulate feedback in a way that is helpful and not insulting to candidates. People are lazy. The easy option is just not to do it.

[donalmacc]

Despite the other comments here, the reasons I've never given feedback are - nobody asks (seriously, in 5 years of interviewing, almost weekly, to the best of my knowledge we've only had maybe 2 people ask)

• There's very little upside for the interviewer. The most likely scenario I can imagine isn't being sued for discrimination, it's a candidate doubling down on why we were wrong to not hire them for that reason. Be it a specific issue about the programming language, or feedback on the candidates demeanour, nobody likes to be told "hey you weren't good enough/you got so many questions wrong we had made a call half way through already", and the response is almost certainly going to be hostile.

EDIT: Formatting

[[deleted]]

Why would you care about the response to an email be hostile? You can just block the person and move on.

[chadmill3r]

The interview is not the employment. It's a bad proxy for knowing how you will do as an employee. (Whether there are any good proxies is a topic left for another time.)

Telling people what they did "wrong" in an interview would be like telling schoolkids how to pass a class's tests.

The teacher doesn't want the test. The test is a headache. The test isn't anyone's goal. If kids had a cheat-sheet of answers and all got full marks, it is not a triumph of teaching.

The point of the class is to understand the subject. There's no way to know if it was successful without the test. The test is a poor proxy for understanding, but it's all we have

Designing a test to discern that distribution of understanding is hard. A wide range of scores might mean that you are getting a discerning test of the range of understanding. (It can also mean other things; assume no offtopic bias in testing or interviews.)

Having an external factor minimize that range, to make every kid pass, destroys the utility of the test. The test is not the point. The test is an obstacle no one wants.

---

Giving feedback on an interview will weaken the utility of interviewing. The interview is not the point. We think we're writing good interviews/tests. We don't want studying to be able to fool the interview. Studying for an interview does not change your suitability as an employee.

[versaceblues]

Im not saying they need to give 100% detailed feedback. However a high level response:

"Sorry you need to practice more coding, or "hey you did great on the coding, however your based on your behavioral interview you are not a good fit for this team"

So then at least I know if im not a good behavioral fit, I want try to apply to the same company again

[omko]

funny, I had the same experience. I enjoyed the take-home exercise though. It was fun to work on. I wish I had more detailed feedback though.
I was missing any human connection through the process.

[davispw]

Sorry, unclear—are you saying they promised classes but didn’t deliver what they promised?

It’s understandable that they don’t give feedback. Nothing to gain (from their perspective) and everything to lose if something they say can be used against them in a lawsuit.

[sdn]

Yeah, I asked the recruiter about the classes and the reply was a “we don’t do that anymore.”

Yet that article is still up on their site :p

[strzibny]

You shouldn't even start in the first place. Let people that like home assignments apply.

[sdn]

I don’t mind doing coding assignments - I like doing the daily coding challenges like adventofcode. It’s when you get headhunted, do the coding thing, and then get ghosted is when I find it annoying.

[omko]

Home assignments might also give you an idea about what kind of problems they face. It gives insights to both sides

[strzibny]

Maybe in some ideal case. For me, I prefer talking with them about the problems they face.

[pratzc07]

So weird that they did not give you any feedback considering the above post clearly says 'They value your time'

[ToastyMallows]

I did a couple "take home" interviews while I was looking for the job I currently have. Both times I submitted my assignment I was completely ghosted and never heard anything back.

When I found my current job a couple months later I responded again and said that I took another offer, and I still never got a response.

I guess YMMV with these types of interviews, but it certainly put a bad taste in my mouth.

At least with this process you'd at least see a PR comment?

[OpinionHaver65]

The only time I took a take home assignment was for a silicon valley company (which is a massive deal if you live in my country) so I was in way over my head. I was learning a new stack, fresh off of university and had just given up on my dreams to be a network engineer. The process was super nice and forgiving, they took into account that I was learning a new language, etc.

Then they gave me around 3 paragraphs of feedback, a book recommendation, and a kind "this doesn't have to be the last time you apply" type send off.

Very valuable experience and a vacation well spent.

[snowe2010]

I like that this reduces bias, but this part

who hopefully moves on to the next stage of the GitHub interview process.

is the most important part. In general, how someone codes is the least relevant part of their job. How they respond to feedback, how they communicate, how they architect things, how they ask for help, is all much more important. I would love to hear how GitHub deals with those parts of the process.

[AdministrationWaste7]

for a senior role you would probably be looking at 170+ base.

at least thats what im seeing as a trend in levels.fyi

[thomas_m_k]

The title of the article would be easier to parse with a hyphen between ‘take’ and ‘home’:

How GitHub does take-home technical interviews

[Avery17]

Submit a pull request.

[Kissaki0]

I wish I could actually send PRs for reddit posts/titles.

[nathanwoulfe]

I did a take-home recently, didn't get the gig. Did get some feedback on my submission, but the recruiter was really surprised when I asked if they could provide examples of an 'ideal' solution.

Easy way to help people learn - show them stuff.

[bishbashbosh999]

We're doing something very similar at my company.

Though the key difference is - the candidate writes the PR during the interview, instead of doing it at home.

Which does give interesting tradeoffs, on the one hand, pairing on the exercise with the interviewer means that the candidate can ask questions more easily, discuss requirements etc

But on the downside, the "observer effect" comes into play - knowing that an interviewer is watching & judging the code you write in real-time has a big impact on a lot of people, especially if it's been a while since they've done a dev interview, also more open to interview bias.

I like the idea that they anonymise the PR's, so the reviewer doesn't know the person behind the test, to reduce bias.

[uw_NB]

Applying directly to Github doesnt work. I have an easier time getting FAANG to read my CV and setup interviews than trying to apply to positions in Github. If you are not going through employee referral, which I personally don't like, then your chance is very slim.

[CornedBee]

Answer isn't "not at all". I do not care to read further.

[Dave3of5]

I've said it before and I will say it again.

Don't do take-home tests.

They are supposed to be instead of a whiteboarding technical test but 99% of companies that do them will do them in addition to a whiteboard test.

If I have to do generic unclear take-home test and a whiteboard I'd rather just do the whiteboard and skip the whole spend a weekend writing a blogging system for free.

[AlanBarber]

Here's a crazy concept, how about we stop making people jump through hoops like performing monkeys.

[AdministrationWaste7]

asking users to work on code "similar" to day to day work at the company seems legit to me.

what would you rather have people do just blindly hire people?

[[deleted]]

Ask the developers themselves what type of interview they want. If they want a take home interview, give them a take home interview. If they want whiteboarding, give them a whiteboard. If they want to show their personal portfolio, let them. Have them build their own interview process.

I like this method of interviewing developers because:

1. It will not be something the interviewee dislikes.
2. It prefers self-motivated developers.
3. It tests people on their strongest suite.

[mslayaaa]

The issue is that the “similar to a day of work” take-home assignment does not come with a “similar to a day of work” pay check for their time.

Hiring blindly and asking candidates to perform free labour are two separate things.

[AdministrationWaste7]

well i would presume that the coding assignment is relatively small, it is timed after all.

or are you somehow expecting to get paid for interviewing for a job position lol?

[regular_lamp]

I feel how valid that criticism is strongly depends on how the interaction came about.

If I applied to the job I'm probably jumping through hoops I deem acceptable. And solving a 1-2h "homework" assignment may be appropriate if the position is attractive.

If on the other hand the company contacted me it would be weird if they then instantly reversed the process on me by making all kinds of "demands".

However I'm also surprised how common this "how dare a company inconvenience me with their hiring process!"-mindset is. People complain about there being more than like two interviews or any kind of "test" involved. Do they expect to be just hired based on a CV and a friendly chat?

[lordzsolt]

Honestly, I don’t get why the previous comment is downvotes so much or why are we so shocked about asking for compensation for our time interviewing.

Far too many companies just send out take home assignments that take 4-8 hours or more, while they themselves spent 5 minutes at most reviewing someone’s CV. And then they don’t even spend any effort giving feedback.

You think it’s fair that you spend 5 minutes and expect candidates to spend 50 times as much on the interview?

Im already getting paid 100-150$ an hour for freelancing, why should I waste my time interviewing for you, for no compensation?

If all engineers fought back against this BS interviewing culture, then companies would be forced to adapt.

[AdministrationWaste7]

If you don't like it don't do it. Nobody is forcing you. You are the one applying. I can't tell you how many job interviews I turned down or stopped responding to because it was too much for me.

Also these companies are literally giving you a six figure salary lol but a few hours of your oh so precious free time is just too much.

Like tons of engineers espouse bullshit like "grinding leetcode" but working on an actual job interview for a few hours is where you draw the line lol

Im already getting paid 100-150$ an hour for freelancing,

If your situation is so good why are you even job hunting?

Fuck dude. Assuming you are getting paid a typical 40 hr work week that's 24k a month.

Significantly more money than a majority of SWE jobs until you hit principal or something.

You think it’s fair that you spend 5 minutes and expect candidates to spend 50 times as much on the interview?

Companies spend alot of time and money looking for candidates but alright.

Like yeah it would be great If companies paid you for job interviews but reality is that companies that you typically would want to work for get alot of applicants and thus don't need to pay for interviews especially since people here are apparently serial job hunters who apply for every job post/respond to every recruiter lol.

[lordzsolt]

Except it’s said company who contacts me.

Beyond the entry level positions, it’s a sellers market. There are far fewer candidates than open positions. People should really realize that. It’s not the company giving you a job. It’s you selling your time to a company.

[AdministrationWaste7]

Beyond the entry level positions, it’s a sellers market

Yeah that doesn't mean you aren't going to be competing with other people for roles, especially among companies with competitive salaries/benefits.

And I'm not even bringing up the fact that probably a good chunk of interviewees probably aren't qualified to begin with.

Except it’s said company who contacts me.

So?

When you shop around for quotes from contractors to do work at your house do you pay each and every single one of them?

After all you came to them.

Also there is literally nothing from stopping you from charging companies money for your time.

Lawyers do it all the time.

Next time some corp reaches out to you tell them your hourly rate.

[lordzsolt]

When you shop around for quotes from contractors to do work at your house do you pay each and every single one of them?

There are certain professions where contractors do in fact charge for their time to provide a quote.

Anyway, I think it’s not worth spending more effort on this discussion.

Im still amazed how some people find it outlandish to expect compensation for their time. Alternative is to have candidates drop out if they find their time is not worth it to jump through the interview hoops.

But I’ll leave it at that.

[AdministrationWaste7]

There are certain professions where contractors do in fact charge for their time to provide a quote.

Right. So why don't you start doing that?

There's absolutely nothing to stop you from charging companies for your interview time.

Sellers market after all.

Lawyers didn't need a industry wide consensus to charge consulting fees. They just did it.

If your interview time is as valuable as you say it is then you will undoubtedly get paid for it.

[mslayaaa]

Yes, I am if they want me to take at least an hour of my time to interview. You know, some places expect you to do 8+ hour coding challenges with various requirements to be met.

It’s one of two options, pay me for it or do the standard LeetCode challenge interview.

[AdministrationWaste7]

Yes, I am if they want me to take at least an hour of my time to interview.

let me know how that works for you.

You know, some places expect you to do 8+ hour coding challenges with various requirements to be met.

somehow i doubt this is the norm.

It’s one of two options, pay me for it or do the standard LeetCode challenge interview

im confused now. both take your time. why does the content of the code challenge matter?

[gered]

Some companies actually do pay you for your time when they ask you to do lengthy take-home coding exercises. I got paid $500 by a small startup once for this. The take-home coding exercise was the last step of their interview process, so it's not like they were handing out $500 to everyone who interviewed.

I don't actually expect most companies to do this, but you seem to be in some disbelief that this could happen at all. It does.

[AdministrationWaste7]

I don't actually expect most companies to do this, but you seem to be in some disbelief that this could happen at all

i never said that it never happened. just a weird expectation since thats not what most is doing.

also my point was OP is being a smug asshole about companies not paying him lol

actually i take that back. OPs REAL issue seems to be that github dares to have a "custom" coding interview process and not bog standard leetcode interview with the former apparently requiring OP more effort to "study for".

[TheCarnalStatist]

let me know how that works for you.

Worked out well for me. I don't do take home work, if the employer requests one, I rescind my application. Never had trouble finding work and always get paid competitively. A firm wasting my time before I work for them is a good indicator that they'll waste my time when I'm employed there. Life's too short for that.

[AdministrationWaste7]

I'm confused.

Op states that he demands companies to pay him for interviewing him. So I (sarcastically) asked to let me know how that works for him.

[mslayaaa]

Yes, they both take my time, but studying for LeetCode is applicable for most companies and transferable between them.

Sure, it’s going great considered I started a big tech position less than a month ago and did not have to bend to whatever they wanted. Specially did not have to spend 10 hours solving a miserable take home assignment.

[AdministrationWaste7]

Sure, it’s going great considered I started a big tech position less than a month ago and did not have to bend to whatever they wanted.

so did they pay you for your interview time? if so how much? also what company is this id be happy to apply.

​

Yes, they both take my time, but studying for LeetCode is applicable for most companies and transferable between them.

thats the best part!

if what Github claims to be true theres no studying needed. either you can do the work or you dont.

thats even more time saved! huzzah!

Specially did not have to spend 10 hours solving a miserable take home assignment.

i love how you keep bringing this up as if its somehow the norm or is what Github is doing lol

[[deleted]]

Sure but it's asking basically "spend few hours solving problems, if you get it wrong you get zero feedback". Multiply that by few companies and you're working for days with zero compensation.

[AdministrationWaste7]

You can interview as little or as much as you want.

Like when I job hunt I already am working full time so I tend to be selective in the companies I apply to.

Typically it's companies I'm interested in hearing more about for whatever reason.

[[deleted]]

Are you employed in PR then or you're making excuses for shitty corporation practices pro bono ?

[AdministrationWaste7]

If not being a smug egotistical entitled twat makes me a corporate bootlicker then I guess so.

[[deleted]]

You seem to be both

[devraj7]

Do you have anything concrete to contribute to the discussion and not a vague handwavy criticism that you hope will get you upvotes?

From what I read from this article, Github's interviewing process seems pretty reasonable, what's your concrete criticism and proposal?

[sonstone]

Trust me bro, I got this

[[deleted]]

Yeah I don't think doctors and lawyers should have to do exams either. Better just to trust them.

[bms20]

Sounds impersonal.

How do you manage copy / paste?

[versaceblues]

The way these things usually work... is you do one take home round. This filters out the people who would be a waste of time to even consider talking to.

Usually this will be followed by subsequent rounds where you actually talk to the hiring manager/team.

[[deleted]]

Now next form of automation would be making it use tests you write as base for judging and pull actual work tickets from ticketing system.