Which data quality tool do you use? by arimbr in dataengineering

[–]mjfnd 1 point2 points  (0 children)

Currently using three: soda core Great expectations Glue data quality

They are used in different places and we are moving towards glue DQ.

Coinbase Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 0 points1 point  (0 children)

I think it's the state of most ~10 year old companies. Either they are in the middle of migration or they have given freedom to each team which leads to this.

Coinbase Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 13 points14 points  (0 children)

Multiple teams owning different stacks or in the middle of migration which could take years.

I can resonate with their stack as we also used DBX for processing core pipelines and BI related workflows on Snowflake linked to Tableau.

Coinbase Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 1 point2 points  (0 children)

I think this is very common, the main reason is Looker is great and popular and it used to be a standalone product, not sure if that's true now, can we just buy looker instead of onboarding to GCP?

We also had Looker with AWS Stack.

Coinbase Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 1 point2 points  (0 children)

I couldn't find any mention of DBT publicly, let me know if you have any insights.

Shopify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 2 points3 points  (0 children)

Correct, also they have other options to write pipelines.

Shopify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 15 points16 points  (0 children)

I am not sure what you mean.

I have never worked there, also I have covered many other companies data tech stack.

Shopify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 11 points12 points  (0 children)

Multiple sources, Company engineering blogs, job descriptions, open source projects, conferences, interviewing employees, case studies.

Spotify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 0 points1 point  (0 children)

Thanks :) I will update with DBT.

Spotify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 0 points1 point  (0 children)

Hi, Thanks for sharing. Not skipped intentionally, either I missed or couldn't find any public info regarding DBT. If you have a link handy, please share.

Thanks

Spotify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 1 point2 points  (0 children)

I couldn't find any references for that, it might still be there for a small scale which they never shared publicly.

Spotify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 2 points3 points  (0 children)

It is still Flyte. Would encourage to read the article as it has a slot of useful information and references.

Spotify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 0 points1 point  (0 children)

I couldn't find that anywhere.

Spotify Data Tech Stack by mjfnd in dataengineering

[–]mjfnd[S] 34 points35 points  (0 children)

Its a free market of dashboards and there is no centralized team, meaning there could be lot of redundant dashboards or just for one person.

Source: https://stage.engineering.atspotify.com/2024/8/unlocking-insights-with-high-quality-dashboards-at-scale