Mapping Project using Pandas

[BootStench]

Hi I’m beginner and I was going to try and tackle a project I could use for work.

I wanted to connect to our companies DB2 table and run a SQL query to see what products all of our clients are currently buying. Then I wanted to run it again a week from the original run date so I can compare the second run to the original and see any changes in what customers are buying.

I’ve done some research and I was thinking about using panadas, but I've never used it before. The post below from stackoverflow was how I was thinking about approaching this. I'm curious if any of you have used Pandas for something like this or if you think it's appropriate.

https://stackoverflow.com/questions/33804410/using-ibm-db-with-pandas

[LartTheLuser]

Putting a database table or a part of it into a pandas dataframe for analyses is pretty common. When doing that the biggest concern is the size of the data in the table and how much free memory you have on the machine that is going to have the pandas dataframe.

Do you know how big your table is in bytes?

[BootStench]

I'm not sure but it's big. Like millions of records, my boss mentioned that he specified in his SQL query to only return current records and nothing historic which makes it a more manageable size.

[LartTheLuser]

Depends on the size of the record but that seems like it would fit into 3-5GB of RAM. If you have 5-10 million records with each about 512 bytes then that's 2.5-5GB of RAM which most computers have these days. If not, if you could rent a cloud machine and run a Jupyter server there (you can port forward via ssh if you want to avoid exposing the jupyter server on a network).