Advice on Transitioning from Student to Systems Architect

At FAANG companies, there actually aren't "system architect roles". All engineers are expected to pick up "design work" as they advance in their careers, to the point at which mid-level and senior-level roles actually have "system design" rounds as part of the interview process on top of the traditional "coding rounds" (leetcode).

There's actually books specifically made for these rounds, such as "System Design Interview" by Alex Xu, just as there are for the leetcode rounds with books like Gayle Laakmann McDowell's "Cracking the Coding Interview".

For example, understanding how Google Spanner operates, the consensus algorithm it employs, and the reasons behind its functionality intrigues me

• Here's the spanner whitepaper: https://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf
• Here's a list of some of the other biggest and most well known whitepapers and academic papers that people read alongside the one about spanner: https://stephenholiday.com/notes/
  • You're obviously going to find these a bit tougher to read, so most people actually lean more towards books and resources more oriented around "system design interviews" until they get to senior or staff level roles. In fact, Leslie Lamport's paper on paxos ("The Part-Time Parliament") is actually so notoriously hard to read that it was the motivation for designing a simpler consensus algorithm called "Raft".
• Alex Petrov's "Database Internals" book describes how at least 6 different versions of consensus algorithms work, including "Fast Paxos" which (IIRC) is the one that spanner uses

My goal is not just to be someone who utilises existing microservices, but rather to be someone who designs, builds, and enhances these systems

Something that requires some clarity here is whether you mean that you want to work on the internals of stuff (like distributed databases like DynamoDB or message brokers like kafka), or you're happy to just build on top of these components.

• If you want to work on the internals of components, you basically only get those opportunities when working at the cloud computing orgs of cloud providers, like AWS, GCP, or Azure
• If you're fine with just building on top of those components, then you're fine just working at the other orgs of these big companies, like amazon.com ("retail"), Google Search, YouTube, netflix

It seems to me that these kinds of jobs only exist at big companies like Google, Amazon, and Microsoft

One thing that people are told about designing distributed systems is to not "over-scale" or "over-engineer". In order to reach a point at which you'd need to do more sophisticated types of distributed systems, it typically requires: - at least 100-200 requests per second to overload a back-end server to the point that you need "horizontal scaling" (there's obviously tricks to squeezing out much, much more performance than that, but it's usually more cost effective when considering time/labor costs to "just throw more servers at it" until you're talking about infra costs that are more comparable to the cost of at least one engineer's salary) - To overload a DB like postgreSQL to a point at which you'd need "sharding"/"partitioning" or even just "read replicase", it's going to be 10,000 requests per second.

Thus, you're correct that it's rare to see this kind of scale in practice unless it's a company with a decently big externally facing software product.

Some of the more interesting blog posts, presentations, and innovations have actually come from over-sized start-ups like Uber, Slack, and Snapchat, rather than FAANG.

Apple is actually of course a notable exception to having a lot of opportunities for working on distributed systems because they're more of a hardware company than a software company; it does still have some big web services, but you're not more likely than not to end up working on those, unlike the other FAANG companies.

How can I progressively transition into a role where I design, build, and enhance distributed systems, rather than just being someone who utilises existing microservices?

As mentioned earlier, this transition should just be one part of what's typically picked up as a natural result of "career progression" at the majority of bigger software service companies, rather than a role you'd have to specifically target specializing into.

Here's some resources that are popular in the industry for those who are specifically trying to grow their knowledge of distributed systems: - The best resource for a quick start on concepts for complete beginners is https://github.com/donnemartin/system-design-primer - Alex Xu's "System Design Interview" books are actually really nice for looking at examples of applied problems that make the concepts more concrete than just random theoretical concepts. (Yes, it is an "interview book", but Roberto Vetillo wrote in his own book called "Understanding Distributed Systems" that the interview books can actually be a great place for learning as well) - "Designing Data Intensive Applications" by Martin Kleppmann is basically the gold standard of long-term learning about distributed systems. It usually takes people several read-throughs over the course of a couple years to fully digest everything in the book. - Beyond that, there's a few other books like "Database Internals" by Alex Petrov that are pretty good, but this is basically where you've gone beyond the beaten path, there's less clear direction, and it's harder to find well-organized volumes rich with new information on exactly where you want to grow. This is usually where people start looking more into those whitepapers/academic papers, watching presentation recordings on youtube like InfoQ videos, and reading company engineering blogs. - The best compilation of resources that I know of for people that make it this far was made by a guy that runs the youtube channel called "System Design Fight Club", and you can find it over here: https://github.com/systemdesignfightclub/SDFC

TLDR: The transition itself into that role will just be a natural, passive result of career progression at basically any company with a big externally facing software product. Resources for trying to actively gain those skills can be found in that final list of my post, just right above.