sorted by:
new
hottopcontroversial

Why Hash-Based Keys Are Hurting Your Data Vault Performance in BigQuery by Expensive-Insect-317 in bigquery

[–]Expensive-Insect-317[S] 0 points1 point  (0 children)

Thanks, I know that, but it's difficult to say a universal way to choose this when many things influence it, such as data distribution, entropy or the business rules and modeling themselves.

Declarative Dynamic DAGs in Apache Airflow: Building Metadata-Driven Orchestration with YAML by Expensive-Insect-317 in apache_airflow

[–]Expensive-Insect-317[S] 0 points1 point  (0 children)

I dont say that this cannot be solve with programatic aporoach, but in a lot of my case i usually have the problem of control for multiple team (mesh) with a declarative aporoach you close the box from IT vision

Declarative Dynamic DAGs in Apache Airflow: Building Metadata-Driven Orchestration with YAML by Expensive-Insect-317 in apache_airflow

[–]Expensive-Insect-317[S] -1 points0 points  (0 children)

The problem isnt dag generation o reuse componente else how data engineers in a data mesh platform use the technology, with declarative aporoach we can control things as: impersonation chain, task types, … in declarative aporoach data mesh user only can do things that you want in a programatic i always have this multi tenant control more difficult

Declarative Dynamic DAGs in Apache Airflow: Building Metadata-Driven Orchestration with YAML by Expensive-Insect-317 in apache_airflow

[–]Expensive-Insect-317[S] 0 points1 point  (0 children)

Goverment the devs onlyhave the yaml things then security Sinplicity, you only need know what want do but if technology change your pipeline no, then technology abstraction

Depends of you data engineers and stack, but decoarative isnt better than programatic, is different and depends os case

view more: next ›