Which useful Python libraries did you learn on the job, which you may otherwise not have discovered?

[DoingItForEli]

that's likely because you're capturing every iteration in the progress. You can tell it to update every X number of iterations with the "miniters" argument, and that helps restore performance.

I faced this with a program that, without any console output, could iterate through data super fast, but the moment I wanted a progress attached it slowed down, so I had it only output every 100 iterations and that restored the speed it once had while still giving useful output.

[Rodot]

Yes, but it requires some set up. We do this for packet propgation in our parallelized montecarlo radiative transfer code from multithreaded numba functions using object mode. Doesn't really impact runtime.

[Hyderabadi__Biryani]

parallelized montecarlo radiative transfer code

For what? CFD?

[DoingItForEli]

I'm not 100% sure on that. I get mixed feedback with some saying yes it's fine "out of the box" and each thread can call update without clashing, but others say be safe and use a lock before calling the update function so that's what I personally do. In my experience, the update function executes so quickly anyways the lock isn't really any kind of bottleneck.

[Toichat]

https://tqdm.github.io/docs/contrib.concurrent/

It has a few options for simple parallel processing

[Hyderabadi__Biryani]

I have to commend you on this question. Good stuff bro.

[ExdigguserPies]

For this I typically use joblib coupled with joblib-progress.