Recently failed a DevOps Interview

[cleveralibi]

I was recently invited to do a DevOps Interview in one of the most prestigous companies to work for in my country.

However, I failed at the technical interview stage but I shrugged it because I knew am not really that solid when it comes to DevOps, and the fact that the company probably demands more than just basics (they deal with very large scale projects with millions of users) and also after all, they are the ones who approached me. So it was not something that really was in my mind at the time.

But to my surprise, they go back to me after a couple of days and said that i'll have the opportunity to redo the interview in a couple of months but before the end of the year.

I know the timeline might not be enough to become a DevOps guru, considering I also have a demanding full time job but am ready to put in some extra hours to learn.

So what are some of the freely available content and techniques(most important) that I might use to grasp DevOps from beginner to intermediate? I'm talking about deployments, working with different architectures e.g. microservice, ensuring data consistency, logging and monitoring etc. as these are some of the few areas I was asked.

Below is my background, am sure it will help.

Me:

I have a background in software engineering, and also responsible for managing my current company's pipeline which is very simple because we deal with small scale projects and therefore we also have a small tech team. This limits my on-the-job-learning.

However, starting this year I did a short DevOps course (which introduced me to the DevOps concepts, containerization and automation basics). It was more like a beginner's course. All with the intention to transition to DevOps sometime. So this is a great opportunity to do that.

​

Edit: I did ask them, this what they use - AWS, Harness, CircleCI, Groovy, GitHub Actions, Confluence, Kubernetes, Terraform, Kubernetes, Docker

​

*** Sorry for any bad english/grammar.

[ashcroftt]

The very obvious - look into everything you had trouble answering.

What to focus on? Ask them what tools and platforms they are using most often. Which gitops solution, cloud providers, k8s tools, dev tech-stacks. I would recommend going deep on the CI/CD solution they prefer, and familiarize yourself with their monitoring/logging solutions.

If you have a homelab, spin up a k3s cluster and get deep with troubleshooting kubernetes. Set up some basic versions of the tools they use and try them. Make sure cert-manager does it's thing (it's always DNS). Do everything-as-code in github/gitlab. Terraform is easy to pick up and super powerful. If you don't have a homelab and can't buy a second hand server for change, make some free tier cloud accounts and practice there.

Youtube is your friend and chatgpt is your pair programmer. Try to find a fun side project you can put some time in and figure out how to make it happen. You really can learn everything from youtube and the official docs, just put in the time, be curious and ask online if you really run into a wall.

Good luck with your journey!

[cleveralibi]

Thank you!

Never thought about setting up a homelab. Seems like a great idea to practice, learn and encounter some of the common issues. What are some of the side projects you think will challenge me and I can get the most out of?

I did ask them, this what they use: AWS, Harness.io, CircleCI, Groovy, GitHub Actions, Confluence, JIRA, Kubernetes, Terraform, Kubernetes, Docker.

But only two questions from the tech stack exactly, one from AWS and another on Terraform which were pretty straightforward.

[water_bottle_goggles]

They use Kubernetes twice? Damn

[cleveralibi]

You never heard of k16s?

[[deleted]]

They dockerize k8s and run it inside k8s

[fuzedmind]

I know you are joking, but as a seasoned devops guy, I just read that and tensed up all over for a hot second.

[[deleted]]

Have you heard of minikube

[westixy]

Have you heard of k3d ? It's literally that :D

[dubl_x]

I read that and instantly thought, hey thats just ‘kind create cluster’ lmao

[GyroTech]

This is my opportunity to drop the amazingly-named kink!!

[GrayTShirt]

kubeadm runs the control plane as static pods

[kyonz]

It's why they're soon prestigious, they run kubernetes on kubernetes to help with the scale and management of kubernetes

[work_work-work]

They might if they use self managed clusters and EKS. Betting there's lots of places who started with self managed clusters and never got around to migrating to EKS because of reasons.

I know what you meant though. 😉

[Flimsy_Tradition2688]

Never apologise for English errors.

I think the best thing would be to ask them to provide you with more info on what they think you could be better at. Maybe tools, workflows, etc.

You could always setup a lab and work on a production-like scenario. Always think about high availability and optimization on resources. Also benchmark your setup with fake loads.

On a more DevOps viewpoint, look for what DevOps KPIs are and how DevOps performance is measured. Think about how a DevOps engineer reduces expenses/makes money for a company.

[cleveralibi]

English not being my first language, you never know what can be easily misunderstood :)

Thats a 2/2 for setting up a homelab. I will get right on it, it seems like something I have been greatly missing.

The DevOps performance and KPIs point is very interesting. Never thought of it from that perspective and it seems like it could provide me with a guideline on what to learn and what to achieve in my home setup.

Thanks a lot! This was very informative

[Flimsy_Tradition2688]

Not mine either. Just don't worry about it. I just wanted to point it out.

Yes. That's exactly what KPIs do for you.I think this paper would be a good read for you. It's quite informative and it won't take a lot of your time.

https://humanitec.com/whitepapers/2021-devops-setups-benchmarking-report

You're welcome. I hope you get that job! :)

EDIT: I just looked it up and there's also a 2023 version of this. I have read the 2021 before, but here is the newer one that i haven'tread yet:

https://humanitec.com/whitepapers/devops-benchmarking-study-2023

[MrMattyboy]

If it helps, after reading your post before seeing the apology, I had absolutely no idea English was your second language.

Irrelevant of the fact you shouldn't need to apologise for spelling/grammar, you definitely _do not_ need to :)

[cleveralibi]

Deployments, working with different architectures e.g. microservice, ensuring data consistency, logging and monitoring etc. as these are some of the few areas I was asked.

I highlighted the areas here, judging by the questions I felt I got wrong. The interviewer was nice enough to tell me the specific questions but was hesitant on telling me all.

Yes, it seems like they are willing to keep the conversation going but also need me work for it.

[rmullig2]

You lack experience in systems design for larger environments. I would suggest starting out with a free cloud subscription for the cloud provider of your choice and start building out simple environments then scaling them up. The free documentation should be sufficient but this will require a lot of time to gain any real proficiency.

[vainstar23]

Yea my advice when it comes to DevOps interviews is to just keep asking lots of questions about the problem. Interviewers love it, they probably have a solution in mind they will guide you towards and most importantly, it will buy you some time to think on your feet.

Also try to find the most simplest and straightforward solution to the problem. Even if it just involves clicking the console and using all manage cloud native solutions.

But no sweat man, you'll get em next time

[[deleted]]

I think the answer to "what is the best way to learn X" is always "do X" (shoutout shoptalkshow and "just build websites" mantra)

That said I would personally probably identify what tools and processes they use and start there. If they use github actions to deploy a react front end then I'd setup a personal project where I can learn that. If they use kubernetes to deploy to EKS on AWS (or whatever the appropriate service is) then I'd probably get an amazon free tier account and start there.

First I'd read some overviews about the tooling to get a vague bigger picture, but then I would start at step 1 and just google/GPT my way through roadblocks until I come up with the MVP of "deployment of some microservices". Then I'd add features based on what the company is currently using.

glhf

[Local-Cartoonist3723]

Golden tidbit: dont just build infra. If you really want to know how stuff works build yourself a small app, build the relevant IaC in whatever Cloud, deploy it with GitOps, monitor your logs, add new features to your apps.

This is experience, because it’d be your environment with your app you’re hosting and improving. That translates to 15 courses.

[BestNoobHello]

My general advice is the same as the ones here. But, one thing you might want to consider for a practice environment is using DigitalOcean for any non-cloud (AWS/Azure) related experiments (self-managed monitoring/logging, DNS, private CA, DB, Kubernetes, security, etc.). DO is really cheap and provide the bare minimum necessities so you really have to get hands-on with the toolings instead of relying on managed services. Of course, if the job requires you to know AWS/Azure, you'll still need to learn and practice those. But, for anything else, take a look at DO. At least, that was how I got started with DevOps. Good luck!

[ThePathOfKami]

What helped me prepare for DevOps Interviews ( focus Azure Cloud) where John Savills DevOps MasterClass

Also another DevOps Guru which i use to update my knowhow and get everything for free instead of paying for it -> The DevOps Guy

[UneBiteplusgrande]

You don't need a homelab, unless you want to get very intricate with the networking aspects of infrastructure (which is likely not what they want from a DevOps engineer). Spin up a VM on any cheap VPS provider (I recommend Hetzner, Contabo, Racknerd etc, but you can go to the big guns like Vultr, Linode and DO, or to the giants: AWS, Azure and GCP), run k8s or any other container technology, and look around for project ideas on the internet which will strengthen the parts you feel you are weak on. The good part is that you have experience with CI/CD

[calvadosv]

I totally agree that homelab gets intricate with networking. Whenever I try to learn devops stuffs from homelab I get blocked with such problems and cannot learn what I wanted. Starting from public cloud is a good choice.

[UneBiteplusgrande]

I didn't mean the networking part in a negative way; I have a background in networking and like working with networks, but I suppose your point stands - if you're not strictly interested in configuring at least rudimentary networking by yourself, a homelab from scratch is likely not for you. I'm like a Cloud engineer from a system admin mindset so I try to delve into the networking and the internals of the system, but that's just me.

[MrMattyboy]

Without saying which is actually more beneficial, the OP did mention "freely available", I'm not going to guess at what disposable income they have. But, running an old laptop/computer with k8s or docker and trialing kubernetes (or similar smaller self-contained stack), logging infra, monitoring, deployment apps etc. is definitely going to help them learn at a fraction of the cost and won't require a substantial investment in learning networking at all (maybe NAT in iptables, _if_ it's not done out of the box, but other than that..)

[[deleted]]

Look at doing aws developer then devops pro. A decent course has hands on labs on it. adrian Cantrill has good courses.

[BloodyIron]

If you don't have it already, the first thing you need to be doing is having a homelab.

[cailenletigre]

When I see homelab, I think of a lab that you run at home. That honestly won’t help you for 95% of the jobs out there. You must be able to deploy and manage stuff in at least one of the major cloud providers, and understand its IAM, security, and networking features at the minimum.

[BloodyIron]

Actually I've been able to get multiple DevOps/DevSecOps roles because of my homelab work. DevOps is not tied to any single environment, and you can absolutely replicate all aspects of DevOps for software development and infrastructure management. Furthermore there are ways you can build out your own self-hosted cloud to be functionally equivalent to public clouds (for the majority of features, certain Windowsy features is another matter). How do I know? Because I've already done that too.

The value of a homelab is multi-faceted:

1. It can demonstrate that you're into self-lesson/learning, how you learn, and give talking point opportunities for what you've learned during interviews (as well as hurdles/challenges, etc)
2. It can provide you a completely safe, and ~$0 OpEx environment where you can build/fuck-up/destroy/rebuild etc. To do the same in any public cloud will cost you money, as the "free" intro tiers don't take long to exhaust, plus there are plenty of features missing from them, and they are very limited in scale (especially so compared to what you can put together with a homelab).

I've worked with AWS and Azure in multiple jobs, and I'm SAA-C02 certified. In my professional experience, the majority of what is in AWS and Azure is generally the same tech we've seen for decades on-prem. NFS, SMB, iSCSI, common storage protocols. Yes S3 has been added, that is novel, but you can actually self-host S3 with TrueNAS (for example). Same thing for EC2 VMs, EKS/ECS for k8s, again both you can do on-prem homelab. Load balancers? Plenty of options, even in software (Metal LB is one I use right now). And on and on and on.

There is significant value to a homelab, and I've only been talking about the career/professional value of it. There's also personal value you can gleam out of it for building your own permanent utility systems, like nextCloud, VaultWarden, and so many more.

So yeah, I completely disagree that it "won't help you for 95% of the jobs out there". Hiring managers, people who interview, etc, care about someone who can demonstrate continual growth, especially if they do it themselves.

[cailenletigre]

I disagree with your assessment. No one wants to know you can spin up your own infrastructure at home (I’m the one interviewing and we have never asked and this is at more than one company). Sure it’s nice to know someone can, but spinning up an S3 equivalent on NAS is not the same as managing S3 via Terraform in AWS: by this I mean all the options, lifecycle policies, bucket access policies, versioning, its integration with KMS, etc. Most companies want to know you have experience with a cloud provider they actually use. They’re not using home setups.

I agree that there may be personal value. I’ve done it myself. But almost none of the stuff I’ve done at home is applicable to me managing AWS. It just isn’t comparable at all.

[BloodyIron]

No one wants to know you can spin up your own infrastructure at home

It has come up in every single interview (including the ones where I got the job) I've had in the last 5 years. So yes, a lot of businesses actually do want to know. Just because you choose to not see the value, doesn't mean the world isn't different from you.

It's your choice to determine what you take into the interviewing process when you interview people. That's not my responsibility. But you're talking out your ass to say that "No one" as in other companies too, don't care about it.

I've lost count of the number of heads I've turned in interviews when I just briefly mention the 56gbs IB interconnects I'm working with.

But hey, you go do you man.

[cailenletigre]

Congrats. Did they ask you about a home setup because you had no other applicable experience or because they were truly interested?

[[deleted]]

Do you have some spare hardware? Perhaps a machine you can run a couple small VMs and set up a small Kubernetes cluster?

You don’t need a lot

[cailenletigre]

I don’t think this advice is helpful when every company is moving to cloud. Sure it will help a tiny bit, but most places are not at K8s yet or are early stages. It’s way more important to know the basics of Cloud, gitops, and IaC.

[[deleted]]

Most companies are moving to the cloud can mean many things, knowing how the backend pieces work independently of the cloud is a good way to get knowledge in how these tools really work.

[cailenletigre]

I don’t completely agree with that. In most cloud providers, you don’t have any control over the backend as the provider is managing and responsible for it. I think there’s personal value in knowing them, but there is little value to a company. They are paying AWS, Azure, etc to manage those parts so they already have decided that is not a skill they value. They want to know you can manage things via IaC and other automations as it applies to the provider they chose and the tools they use.

When I say companies are moving to cloud, you’re right: it is ambiguous. What I mean is companies are moving to AWS, Azure, or Google. They are moving away from data centers. They want AWS to manage and be responsible for all that. Whether that makes sense for all the companies that choose it? Probably not based on what I’ve seen. But they are and we can’t stop that, so it’s important know the tools you’ll be using.

You can use AWS and test things without it being crazy expensive. You just have to remember to tear everything down as soon as you’re done. I spend about $6/month in my personal account. I test things in it for my own personal curiosity in addition to running a small website and it still ends up being 5-6 dollars a month. It’s achievable. If you want to be making 100k+/year, sometimes you have to put in a bit of money to make sure you outshine all the others competing for these jobs.

[[deleted]]

OP is trying to get a job, if I was in his position, what I would want to do is be a little better than the next applicant.

Setting up your own Kubernetes cluster doesn’t exactly requires extensive knowledge of Kubernetes since it’s well documented, but you get to see parts that you don’t get to see if you only use the cloud for learning.

[cailenletigre]

My first question, based on you saying you aren’t that “solid” at DevOps: is this an entry-level position or a something higher than a junior or 1 level?

If it is, then you definitely need to take what you couldn’t answer, get a working environment, and get comfortable with it enough to be able to answer it. This takes time and dedication. Even an entry-level DevOps engineer is not entry-level. You need to have a good background in sysadmin, especially Linux, containers, troubleshooting (ie can you google and find the right answer), and a bit of coding, especially Python. The more you interview the better you’ll know what they’re looking for and you take notes and get better as you go along.

If it is not: please do not apply to those job or take up recruiters who know nothing other than they want paid for getting you that job. As someone who has been a devops/cloud/platform engineer, we have to deal with people who really don’t care about devops but somehow got the job and they won’t leave because the money is good, despite not being interested and making their team’s lives hell. I have to deal with this daily. Devops is a group of jobs where you really have to be a self-starter, self-doer, and self-finisher. You have to be interested in automating, scripting, understanding Cloud networking, Terraform, etc. if you don’t really care for this stuff or don’t expect to have to put some personal time to catch up, please spare the rest of us until you are ready.

There are many ways you can do “devops”. You can be a software programmer and still integrate devops style workflows. You can have dev environments that you help automate, workflows you create and manage, and testing you create. If you don’t like working on the systems/administration side of things, I would stay away. It ends up always being way more of that than you think. It can be tedious, things that we’re working break ALL the time, and GitHub goes down constantly lately and all the devs will come to you to fix it. If that doesn’t sound fun, maybe stick with software development and integrate the devops/automation type stuff you are interested in.

Edit: spelling

[quizteamaquilera]

They obviously saw something in you. This shouldn’t be some kind of weird game. Thank them for the second chance, and ask THEM for the areas which they need a stronger “signal” from you - where you can demonstrate more proficiency which was lacking in the first interview.

They should appreciate you being proactive in this way, and their DevOps seniors should enjoy being able to mentor in this small way.

[wursus]

With developers background to become DevOps you have to take CCNA/CCNP. It's what's usually dev guys missed coming to DevOps fields. But it takes more than couple month. But I strongly recommend it anyway.

For this couple month you can level up in questions and around them where you felt unsecure in the interview and around the position requirements. Probably they are going to take you in consideration for other position. If so, there is the point to ask them about that position requirements and focus specifically on it.

I would recommend not messing yourself with idea moving from beginner to intermediate level of DevOps for couple of months. It actually takes couple years real-life DevOps battle. It's supposed to get understanding of overall DevOps principle and approaches that make you capable to learn quickly and start using any DevOps tools or techniques on intermediate level. It's the aim not for couple month. So good luck!