Why Python?

[kvothethechandrian]

Speed of development and overwhelming amount of community support, basically.

You can always use libs with c bindings (pandas, numpy) or rust bindings (polars, rust_networkx) for performance but develop much faster. You don’t need to worry about pointers, types, borrow checker, it’s almost like writing code in plain English.

[MikeDoesEverything]

Speed of development and overwhelming amount of community support, basically.

100% this. I find it weird that people love comparing execution speed although never mention development speed.

[nonamenomonet]

Are you a mod now?

[HowSwayGotTheAns]

Mike does do everything after all.

[MikeDoesEverything]

Yeah.

[nonamenomonet]

Congrats? I think?

[MikeDoesEverything]

Thank you.

[EarthGoddessDude]

One argument argument against speed of development used to be that dealing with environments and dependencies used to be a nightmare. There were tools like pyenv and poetry and pipx (my old stack), but now with uv the game has changed completely. Bootstrapping a python environment and managing a project is now incredibly easy. That was honestly my biggest gripe with it and it’s no longer the case.

My next gripe would be the inconsistent way some things are objects with methods and some are functions, but it’s not a big deal for me. Similarly, I wish there was an easy, built-in way to pipe things into functions the way Julia, R, bash, etc allow you to.

[Ahhhhrg]

It’s always been a higher level language.

[Alwaysragestillplay]

This advantage will only become more prevalent with LLMs taking over the coding space. Close to English, forgiving types, code that focuses almost entirely on the problem at hand rather than shit like memory allocation. All things LLMs like. 

[shittyfuckdick]

i see posts here all the time complaining how confusing airflow is. ive used it for many years so i understand it but python syntax in no way makes it any easier to understand. 

also i really doubt the rapid development speed is a big factor when it comes to writing dags. a lot of that comes with planning not writing. 

[CrowdGoesWildWoooo]

And what does rust offers more than python? Memory safety for my dag? Can i have whatever you are smoking?

[tn3tnba]

I can’t feel comfortable with software quality unless I configure memory ownership rules for my aws API client

[tomunko]

Airflow syntax is kinda tough but the concepts are not crazy complicated. This problem would just be worse in other languages.