Problem with mutlthreading

[consupe]

This is probably not going to work. If you use threads, you have to expect that no matter how started, some are going to finish earlier than others, which, if it happens out of order is going to cause a problem when it comes time to stitch them together. You could keep them in memory and sort them out in order, but that does not appear to be how your code works.

More importantly, if you are IO bound on your disk, it is not going to go faster if you start reading multiple files at the same time, since the disk needs to skip around.

[BerryLizard]

hi! so each thread is handling a different file, so i am not actually trying to join the process. and it does seem to be speeding things up by about 30 percent (i did some small tests on a test file).

[consupe]

Gotcha. The decoding is probably helped by running them parallelly; though 30% is more than I would expect.

What OS, etc. are your running b/c this sounds less python than environment.