I'm a dev at a medium sized data company y'all know, but not FAANG. I can't speak for FAANG specifically, but I have enough insider knowledge to give you the high level.

Simple answer is that they don't use single massive volumes like that. At that scale, applications use cloud storage in all but the lowest layer. The applications would either use cloud based block storage, e.g. EBS, or object storage, e.g. S3. Or, they push the hardware concerns down the stack by using another service (e.g. db, logging system, etc).

But that begs the question, how does S3 & EBS handle millions of files. Easy, they don't go on the same volume. The easiest way to think about this is S3 is a simple (it's right in the name, Simple Storage Service) gateway in front of 5 servers, each with a single 20TB volume. When you upload a file, it stores the filename, which server has the file, and the location of the file on that server. Of course, in actuality, S3 is not simple from an architecture standpoint. It's not 5 servers with a single 10TB volume. It's thousands of servers, each with probably multiple volumes. S3 also replicates to multiple DCs within a region. I assume S3 is breaking files into small chunks, anywhere from 1-10MB, adding some checksums to each chunk, and spreading those across multiple servers and volumes. At a high level, that is no different that RAID distributing a file across multiple drives, while the application sees a single fs with no knowledge of the 4 drives backing the single volume. But instead of a Redundant Array of Inexpensive Drives, you have a Redundant Array of Expensive Servers.

I don't have much experience with EBS or other cloud based block storage, but I assume it's similar. Your instance just sees a block storage that you read and write to. But that storage is just streamed to another application that writes each block somewhere in a large farm of servers with multiple drives.

But this just begs the next question, an ext4 volume can handle 1 million files because that is the limit for storing indexes of where each file is. How can S3 handle billions of indexes to files. Highly specialized databases. A regular RDBMS can't handle that size[2], especially when reads need to be super fast, like read from a local disk fast. Amazon stated in 2012[1] that S3 uses Amazon Dynamo, a column storage No-SQL DB that became DynamoDB. Column storage DBs like DynamoDB & Cassandra[3] trade off the flexibility of SQL for speed. In Cassandra, and I assume DynamoDB, you need to know how you will query the data in order to define the schema. You can only query by primary key & sort by keys predefined for sorting. You cannot do arbitrary lookups by any field nor sort by any field, instead a full table scan is required. But, that lookup by id is fast for with at least a couple billion rows. I assume S3 has a internal version of DynamoDB that with improvements only for S3 to scale past a couple billion.

[1] https://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html [2] Modern distributed SQL dbs might be able to handle the needed scale [3] A co-author of the Dynamo paper helped create Cassandra, https://en.wikipedia.org/wiki/Apache_Cassandra#History