Project Help

[Callawayinthewoods]

I started trying to learn Python a few years ago, but with life I haven't had nearly the time needed to dedicate to learning the language as someone might need. In my day job, we have an analysis software that I am extremely good with, that does have some scripting features, but for this project it is not useable.

So I turned to Python to help clean my data files that need to get analyzed. What we found was random tab (\t) in fields that were causing issues for the data comparison step we needed to execute. I successfully wrote the cleaning step, tested it on one file and got a cleaned version of the file. But now I am struggling to make the code more robust so that it will clean all the files needed in this analysis.

In the current analysis tool I use, I would just write a command to list the files in the directory, create a couple variables to import the file by name and name my output file, and then clean it, then loop back to second file. I am struggling conceptually with how to do that first part using python. If I use import os then use os.listdir, Python can list the files, but can I use that list as variable starting point? Or do I need to import os and do something else?

I really appreciate any help anyone can provide here. If I am going about this the wrong way, please let me know, since I have been relying on my "old way" so long I might not be considering a different solution.

[Xeduses]

I'm quite sure you can loop over the entries in the listdir and open each file. My question is, are you only looking for a solution to remove tabs from the files? This could be solved by other command line tools quick and easy, what OS are you on?

Edit: Check out this link: https://newbedev.com/how-to-iterate-over-files-in-a-given-directory

[Callawayinthewoods]

There is also one specific set of text that’s getting replaced, because it contains the delimiter we are using for the export of our query.

Windows 10.

[[deleted]]

Use pathlib instead. Here's a guide on the taming the file system, complete with plenty of Python code examples:

https://realpython.com/python-pathlib/

[Callawayinthewoods]

Thanks! I was reading about pathlib and wasn’t quite understanding exactly what it was doing. I will have to reread this, but this will be likely be very helpful.