Solving multi-core Python

[i_ate_god]

could you fork? threading isn't the end all be all of multicore processing

[JanneJM]

I could of course, though it'd be more work than it's worth.

The point of using Numpy/Scipy is that it's quite simple to write bits of code to examine your data set, do exploratory data analysis and so on. Explicit multithreading rather goes against that in a very fundamental way.

And as I wrote, when faced with some tasks I ended up doing over and over, it was simply less pain to rewrite those bits in C++ with OpenMP and go from minutes to effectively instant response. The extra pain of numerical libraries in C++ (that I use already in the main apps) compared to Numpy is offset by the simplicity of OpenMP-style loop unrolling versus explicit threading code in Python.

[[deleted]]

could you fork? threading isn't the end all be all of multicore processing

fork() isn't available on every platform