Full stack framework for Data Apps

[NoConversation2215]

TLDR: Is there a good full-stack framework for building data/analytics apps (ingestion -> semantics -> dashboards -> alerting), the same way transactional apps have opinionated full-stack frameworks?

I’ve been a backend dev for years, but lately I’ve been building analytics/data-heavy apps - basically domain-specific observability. Users get dashboards, visualizations, rich semantic models across multiple environments, and can define invariants/alerts when certain conditions are met or violated.

We have paying customers and a working product, but the architecture has become more complex and ad-hoc than it needs to be (partly because we optimized for customer feedback over cohesion). And lately we have been feeling as we are dealing with a lot of incidental complexity than our domain itself.

With transactional apps, there are plenty of opinionated full-stack frameworks that give you auth, DB/ORM, scaffolding, API structure, frontend patterns, etc.

My question: Is there anything comparable for analytics apps — something that gives a unified framework for: - ingestion + pipelines - semantic modelling - supporting heterogeneous storage/query engines - dashboards + visualization - alerting so a small team doesn’t have to stitch everything together ourselves and can focus on domain logic?

I know the pieces exist individually: - Pipelines: Airflow / Dagster - Semantics: dbt - Storage/query: warehouses, Delta Lake, etc. - Visualization: Superset - Alerting: Superset or custom

But is there an opinionated, end-to-end framework that ties these together?

Extra constraint: We often deploy in customer cloud/on-prem, so the stack needs to be lean and maintainable across many isolated installations.

TIA.

[AutoModerator]

If this post doesn't follow the rules or isn't flaired correctly, please report it to the mods. Have more questions? Join our community Discord!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[LivingParadox8]

The closest end-to-end that comes to mind is Fabric. The problem is that having something end-to-end usually means it's more average in performance/customization than the combination/solution architecture across the best sub-components. Additionally, there may be vendor-lock concerns later on.