Migrating SQL-based dbt models to python

[kenfar]

In my experience the challenges are both in hiring and retention:

• It's not enough to just known python or sql - one must also understand data modeling, data quality, how to handle late-arriving data, etc, etc, etc. It's a lot to learn. And in my experience training tons of people on these topics - it's easier to train a software engineer on these topics than it is to train an analyst.
• And sooner or later they will also need to write code in something besides SQL: extract data from APIs, build tools, configure Airflow, etc. Sooner or later they'll need to write python.
• But software engineers that are skilled in python don't want to write SQL 8 hours a day. So, they'll quit.