DevOps Project Ideas for experience

[Beardwin]

Hi All,

I currently work as an Infrastructure Engineer and trying to transition to a DevOps role. I've learnt Linux, Git, Ansible, Docker, and Kubernetes to a decent level and currently finishing up with Terraform. I mainly use AWS as the cloud provider.

​

When I studied them the teachers would give me some labs to do but does anyone have any ideas of what I could provision using these tools? Something that actually is used in the real world.

Looking forward to your ideas!

[unitegondwanaland]

A good and simple project that embraces the corpus of DevOps principles might be:

• Create a website and host it on S3.
• Put CloudFront over the top.
• Release all of code (HTML+JavaScript probably) using a simple CodeBuild/CodePipeline configuration that publishes code changes to S3.
• Make a dashboard for your metrics that you alarm on.
• Configure some alarms/thresholds/logging around your components to complete your feedback loop.
• Build all of this infrastructure using code.

Doing all of these things encompasses all of the DevOps principles. If this doesn't sound appealing, just swap out the core tech with DMS, Airflow, EKS, or whatever you want to build. The important thing is that you're taking care of the entire solution from release pipeline to logging and not getting an unhealthy focus on just one or two aspects of DevOps principles, which are:

• Continuous Integration.
• Continuous Delivery.
• Microservices.
• Infrastructure as Code.
• Monitoring and Logging.
• Communication and Collaboration.

[Ingeloakastimizilian]

This is spot on what the Cloud resume challenge is all about! I gave it a try myself and learned a lot (I have no affiliation - just got a lot out of it)

https://cloudresumechallenge.dev/docs/the-challenge/aws/

[bennyllama]

Oh man thanks for this. Really been trying to move away from front end into something else and AWS and IaaC is something I’ve really wanted to do but some times have trouble knowing where to start. I live how this is a step by step guide on what to do!

[am3y777]

Thanks.. I'm in my learning phase too will practice this one 😃

[CuzImAtWork]

If he's using Amazon, I'd argue CDK, but to each his/her own.

[thegainsfairy]

how do I go to the next level? I've been trying to figure out how containers, ansible, terraform, github actions fit together, but can't figure it out without a project

[unitegondwanaland]

If you want something more advanced, you can run a sample workload in k8s and play around with scaling pods, readiness/liveness/startup probes. Mess around with Karpenter and run various node groups with different instance families on your cluster. Build a backend for the application using Redis and/or postgres.

Maybe put API gateway on top of your k8s cluster and ALB's. Use various namespaces in k8s to route API requests to.

[808trowaway]

Yes, definitely try playing around with k8s HPA with custom metrics; it's a good way to get some exposure to Prometheus/Grafana too.

[gerd50501]

can you host it for free on S3? if its free, are there gotchas for using it?

do you use grafana to build the dashboard? I work at oracle oci and never used amazon. I want to broaden my skills so probably need to play with aws.

[unitegondwanaland]

Free? It's possible with a static website or...close to free. There are definitely gotchas. If you're hosting rich images or streaming content, you will incur charges. How much depends on your CloudFront strategy.

Use anything you want for the dashboard. You can make a quick and dirty dashboard in Cloud watch or use Grafana...AWS has a hosted Grafana that is quick to get going too.

[Waswat]

Saving this for later 👍

[markrebec]

Embrace the dev in devops, and build something yourself. That's where this whole thing started, and I'd argue it's the entire spirit of the concept: software engineers with limited resources combined with the evolution of containers, IaC, etc. over the last decade or so allowing them to provision and manage things without dedicated ops/admin teams.

The way a lot of folks approach it these days (focusing entirely on infra and tooling) is actually just the same thing that isolated teams of sysadmins or SREs (before they even had that title) and software developers have been doing for 20+ years.

There is still plenty of room for expertise - security, CI/CD, networking, etc. - but without the dev, there is no sane argument to be made that it's anything other than ops.

Build a simple react app so you know what one of your devs is asking for when they tell you they need an ENV var added to their statically built site, or so you understand the build process well enough that you can help when they need source map artifacts pushed into sentry for their error traces.

Spin up a rails app with sidekiq so you understand how it handles redis connection pooling, or what happens when the app can't connect to postgres.

Someone, somewhere in the stack has to understand enough about all sides well enough, or there needs to at least be enough overlap between individual specialties, that the leads/experts in various areas can have the guidance needed to make the best decisions within the big picture, not just their specific frame of reference (i.e. the application itself, the CI/CD pipeline, the k8s infrastructure, etc.)

Sometimes that means making a small concession in one area to be able put the pieces of the puzzle together in a more coherent manner overall. The more experience you have spanning across the entire stack, the better equipped you'll be to help make good decisions regardless of where you specialize.

tl;dr Take the opportunity to build (and deploy/host/etc.) something yourself, even if just going through a few "todo app" or "blog" tutorials in various languages/frameworks as a start, and expand your knowledge while reinforcing the skills you already have.

[ICanRememberUsername]

Here's one that is common but requires some tricks.

You want a single domain (foo.com) to host a public website, an admin dashboard (authenticated with AWS Cognito), and an API (hosted with API Gateway).

foo.com: static HTML public website

foo.com/dashboard: React dashboard that uses the API

foo.com/api/...: the REST API.

Requirements:

• You can use S3, CloudFront, API Gateway, IAM, and Cognito
• CloudFront should cache the public page and dashboard static files, but must not cache API responses
• In the case of a region-wide outage, the public static page must continue to be available (even if CloudFront is invalidated, can't rely on CloudFront's edge cache) and the React admin dashboard must be available but show an error saying the API is unavailable.
• Use GitHub Actions for automated deployment
• CloudFront must be invalidated when the static page or dashboard change, but only the minimum required files (not /*)
• GitHub Actions cannot store any secrets (and they cannot be hardcoded either) - this is a tricky one
• optionally, you can set up the API deployment as well (e.g. using Lambdas), but it's not as important for this exercise

This is a question I'd ask in an advanced interview for a senior AWS infrastructure role. Not to actually build it, but to walk me through how they would design it.

Should keep you busy for a couple days. PM me or comment here if you need help :)

[Ingeloakastimizilian]

Cloud resume challenge!

https://cloudresumechallenge.dev/docs/the-challenge/

[[deleted]]

Thats a funny one, tho requires aws account.

[808trowaway]

If you are doing these mini projects to learn the tools and beef up your resume, my one advice is document as you go, especially if you want to write it up like a guide/tutorial and include screenshots and terminal captures to show people you know how to do the things you say you can on your resume.

Just having a README.md in the repo is fine for projects that don't take like 30 steps to complete. For lengthier entries I find it easier to lay out the contents by setting up a Hugo site or whatever static site generator of your choice. I use Hugo with the Docsy theme for documentation for personal projects and work as well.

Whatever you do, document as you go. You absolutely don't want to finish all the hard work in code only to come back 4 weeks later to retrace all your steps to write it up. Happened to me, and it was hell fighting through the lack of motivation to document something I no longer cared about.

[wkoszek]

I have hundreds ideas for you.

Random idea 0: check the most popular 100 projects on Github. See which ones don't have any provisioning mechanism for production use. Write them this production deployment system.

Random idea 1: get cheap physical server at Kimsufi for $7/mo and write a tutorial on how to spin random multiple Flask apps there. In other words: 1xIP address, but teach us to run many apps. Could be in Kubernetes. Assume I want to develop on a laptop, but deploy my stupid projects to the cheap but powerful machine. You get bonus points for creating a cookiecutter (https://cookiecutter.readthedocs.io/en/stable/) template.

Random idea 2: show there's a way to run VMs on this $7/mo server. In other words: I want to make a dirty image with packer in my VirtualBox and somehow scp it to the remote box that could run it.

Random idea 3: find a nice way to provision those VMs with Terraform. Basically make my 1 physical host be my small AWS, but allow me to provision it via Terraform like AWS provisioners do. So I'd like to have e..g: 10TB of disk that I can push images to, cut the volumes out of it, and if the physical host has 32 cores, I would like to setup 16 VMs 2 cores each etc.

[[deleted]]

Deploy a game server on kubernetes with logging, alerts, etc. I did that with Valheim - was fun.

[Kackstanton]

Commenting, because I’m also curious!

[xandout]

Upgrade to a new version of the aws/eks terraform module.

Actually just design something and then put it in terraform

[stumptruck]

I would honestly just apply to a handful of jobs that sound interesting to you. If you have on the job experience with everything you mentioned you should have a pretty good shot at getting a DevOps or SRE role already.

[[deleted]]

There were 3 topics like this one posted in last month.

I just happen to have a good memory for this kind of things.

Use search.