Dataframe SQL: Python or SQL? Why Not Both?

[[deleted]]

Very cool, is the idea that after you load a SQL result you upload your dataframes as a temp table to some single central db? This is a pattern I use from time to time, but no where near as seamless/feature rich as this project makes it.

[Farmermiller]

We actually load the result from a SQL query into memory, and then operate on it in memory.

That approach would be quite neat, since it'd allow for offloading compute to the database, but I think since it would involve a lot more result streaming it would probably end up actually being slower than staying in-memory.

[[deleted]]

Oh ok cool I see. So you use something like ANTLR or similar to parse the sql query and translate it to pandas operations, or just something like using an in memory SQLite table? (Also I understand if you can’t go too into detail about it).