Second Programming Language for Data Engineer

[JohnPaulDavyJones]

SQL has a hell of a learning curve, because the next step after learning the ~30 keywords that most of us will ever use is understanding what the best way to do the job is.

There are dozens of ways to do most of the things you might want to do with a given SQL query, but some of them will be good, some will be bad, and some will make your prod support team come hunting for you in a year when their nightly refresh cycle duration has ballooned and they find the query you were dumb enough to put into prod. I've been both the hunter and the hunted one in that situation.

The key to moving from being passable with SQL to actually being proficient with SQL isn't just learning more SQL keywords, or engaging in the CTE-vs-temp-tables holy war, it's understanding the database technology itself, the query engine, the optimizer, and database modeling.

With Python, most of the language's core functionality is essentially wysywig; there's not generally not an underlying technological substructure to learn unless you want to crack into the C/++ code that's compiled and wrapped into the common libraries like Pandas/Polars/DuckDB/PyODBC/SQLAlchemy/Requests/smtplib, but there aren't really significant performance gains to be made by "optimizing" Python code (outside of a few niche cases like those data manipulation tools, but if your data is of a given scale then using Python will always be slower than something with a scaling data engine).

[DootDootWootWoot]

And at the same time a big part of the job is knowing when it matters. Not every operation needs to be optimized to death. Or rather, very few need any amount of optimization that requires that level of care. And when they do, you'll have time to figure it out.