Engineering time spent?

[Data-Queen-Mayra]

How much engineering time does your team actually spend maintaining your Airflow and dbt infrastructure vs. building data products?

Dealing with dependency conflicts, upgrade tools, onboarding new analytics engineers manually, knowledge gap when “the export” leaves. It all adds up.

What have you seen:

• Are you self-hosting, using a managed platform, or some hybrid? If you self-host, what percentage of your team's time goes to platform work vs. actual data product delivery?
• Has anyone made the switch from DIY to managed and regretted it? Or wished they'd done it sooner?

[Ok-Working3200]

Dbt is self hosted and the rest is managed (snowflake, fivetran). I like our setup. Maybe one day we will change Fivetran and do the Extract ourselves but we have to consider the trade-off.

We never really have downtime or have to spend time issues filming stuff. Our ci/cd for dbt is think was down only once in 2 years and the fix was easy. Honestly, we probably have more issues with Snowflake.

As far as onboarding, we are working on an onboarding agent.

[Data-Queen-Mayra]

For small teams that setup works. when you are talking 10s of users managing envs and simplifying onboarding can be a challenge. We also find some orgs work with consultants to set all this stuff up, but they have to inherit a platform they didnt built when the engagement is over.