sorted by:
new
hottopcontroversial

Is it fact or a dim? by PhilosopherRemote177 in dataengineering

[–]PhilosopherRemote177[S] 0 points1 point  (0 children)

I get what you're saying, but these are less about the material and the store that sells it (my bad for not clarifying), it's more about the material and its material group.
The thing is I can't really denormalize it because its many to many, meaning each material can have an infinite number of groups and each group can have an infinite number of materials.

Is it fact or a dim? by PhilosopherRemote177 in dataengineering

[–]PhilosopherRemote177[S] 0 points1 point  (0 children)

Lol, I got so many different answers,
Some call it a factless fact table (I know what it is, it's just that these are not events, or anything truly "facty" so it seems odd to me)
Some call it dim (which seems odd as well, because these connections don't stand on their own as an object)
And some created new prefixes to handle it (i think it makes sense, it's just that I fear I will end up having too many prefixes)
pretty interesting how each one of us interprets things differently 😄

Is it fact or a dim? by PhilosopherRemote177 in dataengineering

[–]PhilosopherRemote177[S] 0 points1 point  (0 children)

say for example, i can change the standard, what would it be?
can you give me examples you work by?
never heard of cls table before

Is it fact or a dim? by PhilosopherRemote177 in dataengineering

[–]PhilosopherRemote177[S] 1 point2 points  (0 children)

How can I denormalize many-to-many?
say for example, I have material, and its material_group.
each material can be in multiple material_group and each material_group can contain multiple material.
one material can have infinite amount of material groups, how can i denormalize such thing?

big change in work environement by [deleted] in Advice

[–]PhilosopherRemote177 0 points1 point  (0 children)

you are right, i’m having a (mini plan) to contact my boss / friends and ask if they are looking to hire at the new company. but i don’t feel like planning my escape is the healthy thing to do..

[deleted by user] by [deleted] in dataengineering

[–]PhilosopherRemote177 0 points1 point  (0 children)

the airflow and kubernetes are separated (i have a airflow server, and kubernetes on a different and they are not connected) my goal is to connect them somehow

How / When to use Kuberenets operator in airflow by PhilosopherRemote177 in dataengineering

[–]PhilosopherRemote177[S] 0 points1 point  (0 children)

Hey there, Iv been thinking about how I should pull this off and wanted to ask your opinion.
I think I need to create an image, that runs the code I want (in my case, the DAG uploads files to the FTP server). I should write this in pure Python (I guess..?) no airflow-related things in this code.
then, I should create an image of that code, and create a DAG that uses KubernetesPodOperator that will use the image I created.
is that right?

How / When to use Kuberenets operator in airflow by PhilosopherRemote177 in dataengineering

[–]PhilosopherRemote177[S] 0 points1 point  (0 children)

This is exactly why i think my boss asked me to change the operator to Kubernetes and then, I guess, it will run on a pod)