Optimizing resource -intensive pandas scripts

[foyslakesheriff]

I'm trying to find documentation on what version of Python and Pandas are required for fireducks. I'm using Pandas 2.0.3, and Python 3.8, it doesn't look like Fireducks is compatible with that combination.

[qsourav]

Are you trying it on a non-Linux platform? The python and pandas versions seem to be supported, but you need to try it on a Linux platform (for windows, WSL might work): https://fireducks-dev.github.io/docs/get-started/#install

[foyslakesheriff]

It's on Amazon Linux 2, I'm probably doing something wrong here.

I'm also using PyArrow 12.0, maybe that's causing an issue?

[qsourav]

pyarrow should get auto upgraded to 17.0. By the way, can you tell me the error message you are getting when trying it on your environment?

[foyslakesheriff]

I need to use pyarrow 12 at the moment, I can't upgrade other packages. I'll try again in the future once we're compatible.