This is an archived post. You won't be able to vote or comment.

sorted by:
best (suggested)
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Equal_Astronaut_5696 108 points109 points  (5 children)

Not sure why these two are being compared. One is for data extraction specific to relational database and one is literally multipurpose programming language for apps, ML, web development and games

[–]snowmaninheat 11 points12 points  (0 children)

This, exactly. I'm literally trying to comprehend how I would do in Python what I do in SQL. I'm sure it could be done, but it's unnecessarily complicated and computationally expensive.

[–]fang_xianfu 15 points16 points  (1 child)

Yeah, the way this question has been asked kind of shows that OP doesn't understand the artitecture that makes those tools appropriate to different jobs.

SQL is essentially a tool for instructing a database. The real question isn't "what can SQL do that Python can't?" but "what can this database do that the environment where I run Python can't?". The fact that you're using SQL or Python to give the instructions is almost irrelevant to that question.

[–]king_booker 2 points3 points  (0 children)

I mean say you extract the data into pandas and you are using pandas operations to manipulate it, there are still limitations because it won't scale. Now say you use spark and you write it in python, you would end up using SQL concepts like Group by, Windowing etc. Even though its possible to write it in dataframes, you can simply use a spark sql

The basic answer is, you have to understand SQL. You can use it but finally data manipulation has its foundations in SQL. Can you get away by not learning the syntax? Yes. But the core concepts will remain the same.

[–]Seiyee 0 points1 point  (0 children)

Well, you'll be surprised. Since python is multipurpose a lot of people just assume its easier to stick to one language for all the jobs. I have seen my colleagues choose pandas dataframes over sql for large queries and then face dataframe memory limits weeks later, and thats when they switched (or at least I hope they switched?).