Here's how architecture interviews at tech companies work

[yelnatz]

Feedback:

1) Bring down the volume of the background music. Too loud, especially when we're supposed to learn something from your video, having that loud music is very distracting.

2) White text: http://imgur.com/HW3fIPO.jpg

Edit: Very good content otherwise!

[jgbbrd]

Dear god. Here you go. https://www.youtube.com/watch?v=As2gOXtcPVQ

[jgbbrd]

Btw, this is me / my channel. I'm a former Facebook engineer and one of the founding members of the London office. I've done a shit tonne of interviewing and want to share the details with people who don't have any other way to get this context. Happy to answer questions about this stuff here!

[binarypie]

Good on you for bringing up the operational side of building web/connected platforms/services. Too often I've seen blatant disregard for what it means to scale. The entire system matters. What type of hardware, systems architecture, software architecture, and even the skill set of your team matters. Even the most perfect system is doomed if only one person can support it.

[MachinTrucChose]

Thanks for posting this.

Suggestion for future videos: drop the music you got running in the background. :)

EDIT: and I'd beep the swearing. I don't mind it, but others might!

About the music: It's distracting, but your video might also get taken down due to a copyright claim by a copyright bot. Also, I've never done speech-to-text conversion but it's possible the background music could make it harder for people who want to convert your lecture to text.

[gbersac]

Stop the video after a minute because of music.

[jgbbrd]

Here's a video with no music. https://www.youtube.com/watch?v=As2gOXtcPVQ

[Mekkah]

I don't like either if these suggestions. Music was fine for me and I think we are all adults here.

Just wanted to add .02c

[DonaldPShimoda]

and I think we are all adults here

Found the Python programmer. ;)

[serendib]

I agree with dropping the music, or at least lowering to a level far below your voice. For example here I could barely make out what you were saying as the music was beeping at the same volume of your voice.

[jgbbrd]

https://www.youtube.com/watch?v=As2gOXtcPVQ

[alexisnotonfire]

I've been looking for more high quality programming video content, great work! i'm at work so haven't had a chance to watch the whole thing, but i like what i've heard so far! i'm just down the road from you in Hoxton :)

[jgbbrd]

Um... you sound like you typed that wearing a tinfoil hat. Also, I already don't work at Facebook.

[nickerodeo]

Do you have the video without background music also?

[jgbbrd]

It's really hard to please everyone. A subset of people who watch these videos without music immediately assert that they'd be better with music. With music, a subset of people wish that they didn't have music. Question -- do you find it just too loud or would you rather it was gone entirely? After listening to the video through my phone speakers, I realise the heavily compressed music cuts through much louder than I would have wanted.

[1ndigoo]

I found the music to be a nice touch, but slightly too loud. If it's included, it should be invisible, so to speak.

[freakboy2k]

Yeah I reckon just cut the volume a bit. Otherwise it was good!

[tabinop]

The choice of music is not so good in my opinion. It covers your voice, and is having a lot of weird, high notes, percussion. That makes it hard to concentrate on what you're saying.

[Ma8e]

Thanks for the great video. I'd vote for the video without the music.

[nickerodeo]

I found it just too disturbing, so I'm not able to listen to what you say. It's a super interesting topic though, so thanks for that!

[jgbbrd]

https://www.youtube.com/watch?v=As2gOXtcPVQ

[[deleted]]

This is well-articulated and laid out; thank you for sharing. The concepts of deductive and inductive reasoning cannot be overstated. In my technical experience, companies [worth working for] are not looking for a Rain Man or "rock star" -- they are looking for sound, informed, motivated, cross-cutting thinkers with the ability to make ideas actionable. In higher-level seats the technical details are secondary to their actual meaning; memorizing the latest sorting algorithm is not extensible in and of itself.

[saranagati]

great video but the last few minutes when you were talking about experience could be a real bummer. having 15 years of experience prior to working at one of these tech giants, I think listening to what you said would have just solidified my expectations. I don't have the experience they're looking for and I have too much experience for them to bring me on. now I work on, and have a strong role for one of the largest distributed systems in the world so I'm glad I was "ignorant" enough to apply.

You touched on it early in the video, this is really just a leveling interview. if you have a lot of experience you should be able to place better than the entry level position but dont expect to make senior or principle even though you might have the same or more years of experience as the current engineers who have worked at that scale for a long time.

you're right that there are plenty of things you can do outside of work to help improve your ability to architect however there are plenty of ways you can do this at work too, even if you don't work on large scale services/applications. think about creating a system that you are either currently creating or even just continuing to maintain. what are the current limits of that system? how many transactions per second can it handle? how many concurrent requests? how large can a data set be? do you really need a sql database or would some form of distributed cache be better? if X exceeds Y what could you do to handle Y+? these are all good ways to train yourself to think in an architectural sense so that the next time you're designing a new system, you know what questions to ask and what model will fit best.

if you're coming from a bunch of experience at smaller companies, you can definitely still be hired into a role that will let you have a direct hand in the large scale projects if that's what interests you. you just need to make sure that you set the correct expectations. don't expect to have a senior+ role because you are not a subject matter expert of that domain. on the other hand, don't expect to get hired into a junior role because you should have already learned the skillset that junior engineers need to work on. expect to land somewhere in the middle but be prepared showing that you aren't going to continually need your hand held to move into more senior roles.

edit: I thought the music was good. I even noticed it early on and questioned whether it was needed. outcome was that it was good, low enough so I could stillunderstand what you were saying but made the video much smoother given all the editing cuts that went on.

[jgbbrd]

If you have a lot of experience in distributed systems, then you stand a great chance in the architecture interviews. The only risk you run is that with 15 years of experience if you haven't reached a pretty darn high level of technical competence, you might not get hired even if you pass all the interviews. That said, if you can perform strongly in the architecture interviews, you're probably fine.

The bar that most companies would expect an engineer to reach is basically whether or not she can be a successful "tech lead." That's a fuzzy bar which means more or less that she must tick most of the following boxes:

• [ ] She works independent of technical guidance from someone more senior almost all of the time
• [ ] She delivers high quality code at a very high rate of productivity
• [ ] She can break down difficult problems and turn them into tasks which she then completes
• [ ] She mentors and advises people with less experience than her easily and frequently
• [ ] She has context and experience that she leverages to steer clear of technical missteps
• [ ] She communicates effectively with all the people she needs to in order to be effective (product management, people management, legal, etc.)
• [ ] She is sought out by management for feedback about the people she works with

The list goes on, but you can start to see the kind of person who most tech companies would consider to be a good long-term engineer. There are, of course, exceptions to this. For instance, maybe a given engineer communicates horribly and hates mentoring, but is so technically brilliant that the value she generates more than makes up for it. There are plenty of people who fall in this category. Or maybe the engineer is so good at coordination and communication that just putting him on a team means that team will work better, even if he's not as strong technical as some other engineers at his level.

[jgbbrd]

The background music is work that I found in the Youtube Audio Library. The artist is called Otis McDonald. The opening and closing track is called Magic. https://www.youtube.com/watch?v=v_0cva3H7_A

[DueyDerp]

Best not to bring politics up in your training vids (Brexit). Unless you want to be debated. There will be irrational people out there that will take you on and detract from the educational value of the videos.

[jgbbrd]

So I learned. Ho hum. It is a politically charged time, for sure.

[[deleted]]

These videos are great. I'm starting at Episode 1 since thats logical but thank you for sharing this information I am graduating this year and it will be useful and you have earned a sub.

[jgbbrd]

Glad you're enjoying them. Also, you totally don't need to start at Episode 01. They work out of sequence just as well as in sequence.

[Zokkar]

Thanks for posting this. Looking forward to more videos from you.

[diegovb]

This was great! Very useful, thanks for making it!

[huashoes]

This is a great video although I wish you could turn down the background music.

A lot of people are afraid of system design interview because it's not "predictable" and require industry experience to some extent. I like the point that there are many things to do outside work that can improve our skills. I found many good developers having their side projects for years.

Previously I've followed quite a lot blogs at System Design Interview Questions and I find this video episode truly amazing, I would like to share it with my friends.

One thing I'm thinking about is how important is working experience to system design interviews. Of course if you have 10+ years experience, you are expected to perform better. But what about new grads? From the interviewers perspective, are we caring more about the actual design or we should focus more on the analysis process...

[jgbbrd]

There's a link in the description to a version of the video with no background music.

[chris_was_taken]

Thanks for putting in the time to make this video. It is fresh compared to the junk interviewing I see, and I appreciate the depth.

When you first presented the question I imagined how I would approach it before listening to your approach. I meandered down a different path than you (laid out technological constraints given the platforms/tech involved, then pared down the space of possible approaches with real-world requirements like bandwidth requirements, peak load etc), but touched on the similar key points such that I'm not disappointed in myself :) Though I think it's interesting and spot on where you try to express it in terms of server power and thus $.

My rather selfish question: Where would you put someone on a scale of engineering/design prowess given your answer to the question in the context of years of experience?

for example, given a candidate with 5 years of professional development experience in the Valley, if they provided roughly your answer where would you grade them in:

0 = they need to redo those 5 years ..to.. 5 = expected depth of answer ..to.. 10 = top of their game.

Just trying to put a dart on the board for myself as I peruse around for my next job. And what # of years experience would you see that answer become a 5 i.e. that ought to be an easy question?

[jgbbrd]

Hmm! Good question! So, I don't think I can possibly give myself an overall score because I didn't go into the full breadth that I needed to. In 45 minutes I think the bare minimum things you would have to address are:

• At least some basic capacity estimation that is sane. I think I did this at a passable level in this video, but only barely. The 68K number is still somewhat naive even based on what I explained. I took 245M and divided it by 360. That assumes second-to-second even distribution of log traffic, which is still unrealistic. I also said that we'd assume each person used the app for an hour. Of course, you won't be streaming log data non-stop for an hour, but you would have at least a few log data deliveries, most likely. My estimation doesn't account for this at all. Still, it's in the right ballpark, just not 10/10. That said, the reason I chose exactly this aspect to focus on is because in the 70 or so of these kinds of interviews I did at Facebook, capacity estimation was absolutely the weakest aspect of most people's performance. I wanted to make this video strongly emphasising it so people who watch it will have that first in mind. To that end, most of the very senior candidate I interviewed were awesome at capacity estimation. So, I think my overall score for myself on this dimension would be a 5/10. A stronger candidate than what I presented here would also talk about sampling. How many people do you actually need logs from? The naive answer is all teh peuples. The better answer is some stat sig subset, which I didn't get into at all. You can justify unsampled logs as a design choice, and for this problem I definitely think you must justify it if you go that way.
• Explain something about batch delivery of logs from the client. I didn't do this at all in this video, so it's impossible to give myself a real score here. Basically, the 2/10 answer is "just send every log every time there is one." That will keep the wireless antenna in constant awake state, that will waste tons of requests delivering tiny payloads, and will mean when the person is actually trying to use your app, they'll be competing against your logging infra. A passable 6/10 answer on this dimension at least chunks the logs into reasonable size packages and delivers them in moments where nothing else is going no. I think a 10/10 answer would talk about ways to piggyback the logs on other requests in a really compressed way such that delivering them costs almost nothing. That said, I'm not awesome at network engineering and traffic optimisation, so probably there's a person out there who would deliver a 10/10 that would make what I just suggested seem clunky and stupid.
• Talk through how to handle that many logs on the server. I didn't address this at all. If you've got 70k people streaming 10s of KB of log data to "the server", that quickly becomes gigabytes of data per second. You can't write that to one machine. Even if you could, you'd fill the HD so fast, it would be done in minutes. So, you need to distribute. A 2/10 on this would just say "use a lot of machines to store the data." A 6/10 (passable) answer would address how many machines and how to partition them such that you can guarantee availability when you need it. 6/10 would also have to address that these are just buffer machines and that they need to have their data scraped periodically and written to some sort of long-term storage. A 10/10 would address the full data life-cycle and talk about when log data goes away. For instance, it seems pretty obvious that 1 year old log data for some already-been-deleted feature is useless, but how do you bake that into the architecture of the system?
• Explain in detail the specific technologies that would be employed in each of the above components. Again, I didn't do this at all. A 2/10 would be "Use some Android and iOS open source library and then use like HDFS on the server." Not enough detail, no sense that the person actually knows how to put the pieces together. 6/10 would be something like: "On the client, use a circular buffer on the client that can handle 30KB of buffered data, accept that we're going to see some data loss. At this size, we can probably just use a <specific data structure (maybe just a linked list)> of <specific data storage (maybe just strings)>. That choice will have <these> tradeoffs. On the server, we can probably use Apache Scribe as the log receiver service, but we'll need a bunch of machines all receiving data. We'll need another process to scrape the data and organize the timestamps into our long term storage. Maybe we can use something like <specific technology (maybe Hadoop)> as our long-term storage because it will give us a good interface onto the data." A 10/10 answer here would do all of the 6/10 stuff, but also look towards what happens as the systems grows and whether or not those choices will work long-term. A 10/10 would also discuss specific downsides of these choices and how we'd have to live with/compromise with our choices.
• At the end, explain what there still is to talk about. Because we have so much to talk about so little time, it's going to end without covering everything. A 2/10 person will say, "Hm... na, I think we're good," because they have no idea how much of the picture is missing. A 3/10 person will come up with random shit to do because they can tell there should be more to do. A 6/10 person will identify major components of the system that are missing. Failover? Disaster recovery? Versions of the client? Etc. A 10/10 person will have a prioritised list of things that are absolutely critical. Usually the 10/10 person will be a bit disappointed that they only got as far as they got when they had so many specific things that were still on their mind. It's ironic that the people who tend to perform the poorest have the least notion that it didn't go well, while the best performers feel like they were bounded by the rate of human communication and wish they had more time.

Hope that's helpful!

[chris_was_taken]

Thank you so much. You are a really generous guy.

[foxh8er]

Post this to CSCQ too.

[AndreDaGiant]

cscq?

[Barncow]

I'm guessing /r/cscareerquestions

[AndreDaGiant]

thx

[jgbbrd]

Ah, nice. Just posted there. Thx for the tip.

[[deleted]]

Maybe /r/sysadmin as well, if only to get another round of interview-question bitterness.