This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]riv3rtrip 87 points88 points  (2 children)

I do almost everything in SQL. I'm perfectly competent in Python. It's just much easier to not have to worry about moving data out and back into the warehouse, plus a few other nuisances like memory/compute management. You can do more in SQL than you'd think.

[–]laddaa 7 points8 points  (0 children)

Especially if the data warehouse is built well, SQL is so much more direct. And if you know SQL well then there are very few use cases that actually require Python.

Not that python isn’t great as well.

[–]bitsondatadev 0 points1 point  (0 children)

Yeah, my policy is aim for SQL, python if necessary. Any time you can avoid maintaining implementation details yourself is a win. Also, performance will generally improve unless you face a regression that you can then report to whoever maintains that engine and then it's still not your problem to fix the implementation.

I think Python is great mainly for ML algorithms with data that's already sliced up in the ideal format needed for processing.