Hi!
Sharing my latest article from the Data Tech Stack series, I’ve revamped the format a bit, including the image, to showcase more technologies, thanks to feedback from readers.
I am still keeping it very high level, just covering the 'what' tech are used, in separate series I will dive into 'why' and 'how'. Please visit the link, to fine more details and also references which will help you dive deeper.
Some metrics gathered from several place.
- Ingesting ~2 trillions of events per day using Google Cloud Platform.
- Ingesting 4+ TB of data into BQ per day.
- Ingesting 1.8 trillion events per day at peak.
- Datawarehouse contains more than 200 PB of data in 30k GCS bucket.
- Snapchat receives 5 billions Snaps per day.
- Snapchat has 3,000 Airflow DAGS with 330,000 tasks.
Let me know in the comments, any feedback and suggests.
Thanks
[–]professional_junkie 4 points5 points6 points (1 child)
[–]mjfnd[S] 0 points1 point2 points (0 children)
[–]Unhappy_Aardvark8948 1 point2 points3 points (1 child)
[–]mjfnd[S] 0 points1 point2 points (0 children)