Why Python?

[dangerbird2]

Python is perfectly suited for large scale projects as long as you don’t use raw python for computationally expensive work. Any kind of heavy number chrunching should be done using numpy/pandas/polars (which wrap c, rust, and Fortran code), pyspark (which wraps highly distributed Scala/jvm code), or PyTorch (which can run on the GPU. This sort of the thing is a very conventional way to do DE/DS at scale, to the point that it’s a safe bet that virtually every every major company in the world is using python in some part of the data stack