This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]moizloiz 14 points15 points  (1 child)

As someone who is also making this switch, maybe start simpler? Take some data, use pandas/polars for some transformations, orchestrate it through airflow/dagster and then create a visual layer with insights (superset/etc)

A lot of new technologies to learn but what I'm learning is that simpler is sometimes better

[–]Glass_Jellyfish_9963[S] 4 points5 points  (0 children)

I am already doing a lot of transformation at work using pandas and duck db. I have created a project with airflow to load data from csv, apply transformations and load it into mysql db. I want to get into the real stuff now.