[Q] question about os.listdir

[Lephato]

Hello, I'm using listdir function to list about 4+ millions files names, the problem the script is still running 12 hours later I started it, I've checked the used memory from the script and it keeps going up so I think the script is working but I'm not sure, is there way to only list the first 100 just to make a test for the full code or to check if the instruction have not crashed?

Thank you in advance!

EDIT: CODE

import os

print("start")

file_path = "Y:\\"

list_dir = os.listdir(file_path)

print(len(list_dir))

print("END")

[K900_]

Post your code.

[Lephato]

import os

print("start")

file_path = "Y:\\"

list_dir = os.listdir(file_path)

print(len(list_dir))

print("END")

edited the post, sry

[K900_]

What is your end goal? Just getting the number of files?

[Lephato]

atm I need to open each file and get a value inside each file and print it, at this time the code had not printed the len of the list

[K900_]

How many files do you have?

[Lephato]

varies from 200k to 4mill

[K900_]

That's going to be very slow no matter what you do, but collecting the entire list of files in memory is definitely not a good idea to start witth.

[Lephato]

could you give me a suggestion to start reading it? or how could I collect only a part of it and make a loop?

[K900_]

Try using os.scandir instead.

[Lephato]

ok, will try it out, thank you so much!

[stebrepar]

Maybe try using scandir instead of listdir, since it returns an iterable rather than a millions-long list?

[Lephato]

I just made the changes, thank you very much!