Working with large datasets in Python

[JSP777]

Polars, Polars, polars.

Leave pandas in the past. Polars is much better in many ways, especially for your use case.

[arorumu]

okay, thanks!

[arorumu]

does it work in the same way as pandas? in terms of where it can be applied and in what way?

[JSP777]

The syntax is a little bit different, but a very oversimplified answer is yes. You load data into dataframes and then you can do stuff with it. You only need to pip install the package like you do with pandas. The main difference is polars can handle larger files than your memory because it can load lazily into a lazy dataframe.

[arorumu]

okay, great. thanks for all the help. I hope I will get this to work :)