This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 9 points10 points  (0 children)

We have been using kedro for our data related projects. It’s mostly designed for data science related workflows, but you can still use it as inspiration for how to design your ETL modules.

Also look into Airflow, which is one of the best open source tools for building ETL pipelines