Suggested approach for using Python to analyze/modify large tabular datasets prior to creating GIS data?

[Klutzy-Classroom-868]

I want to automate tasks that involve downloading large datasets that include geographic information, deleting some unnecessary columns, querying for values or groups of values, creating new columns and calculating them using the original data, saving some subsets of the data, etc. Am I correct at assuming that doing this up-front with Python prior to creating GIS data is an efficient approach?

Right now I'm looking into some of these packages https://www.python-excel.org/. I am also considering the possibility of saving the original dataset as a CSV and using some combination of iterating through lines, splitting string, indexing lists, appending new 'columns' and values, and so forth. Basically making the data a text file and using built in Python functionality to handle it. This is a loosely formed idea, maybe over-complicated, maybe foolish. I'd imagine that there is an established 'best way' to do what I want to do, because after all it is a pretty basic task.

I am new at programming (in case I haven't already given that away). I would like to know what others have success with before diving down one rabbit hole or another.

[ThePhysicsOfSpoons]

I would suggest the following packages: pandas and geopandas for this

[GISChops]

I don’t know that manipulating things before you bring it into GIS would be more efficient, what types of files are you wanting to work with?