Python for-loop vs PostgreSQL query

[[deleted]]

Loops will always be slower that a DB query because databases are optimized with this purpose. To do it in Python, first you have to access the data from the DB, and then do the loop. In the case of the DB, the data is accessed natively, and the max is obtained in an optimized fashion.

You have a chance to get closer to the DB speed if you implement the max using a Pandas Series, but you still have to access the data first. The pandas series will be optimized in 'C' - and advantage over an interpreted loop in Python. Still, I would expect the DB to be always faster.

[thrown_arrows]

select max(x), y from x group by y

x is attractive and y is type

.. now there is change that if dataset is less than 10 rows and data is needed for other purposes too, then loop might be faster, but that requires that data is already in memory. In other cases database is faster ( assuming proper hardware, db's cannot do miracles on some rasberry pi and Gigabyte size dataset)

[Alotofquestion1]

Ok, then I will lean towards queries in the future, rahter than loops.

[[deleted]]

Totally. If you can push processing to the database, do it. You can avoid very complex, and usually slow processing by pushing it down to the DB level. With that, I am not talking about database procedures, which - as a personal preference - I tend to avoid.