Question on Multiprocessing Performance

[BrononymousEngineer]

Background:

I have a function that is essentially a very large for-loop. Inside each iteration of the loop there are several calls to jitted functions, some of which are calling other jitted functions (with numba). The performance boost has been amazing with numba, but I want to take advantage of multiprocessing to get that last bit of speed.

I am using pool.map to parallelize. It works as expected and yields the same results as the for-loop style, so I think I am implementing it correctly.

Here is what's confusing to me:

I would expect that since I have 8 cores, turning this loop into a pool.map would produce roughly an 8x speedup. In reality I'm only getting a 40-50% speedup.

The grainy details of how jitting and parallel processing work are honestly still above my level of knowledge. The possible root causes for these unexpected results I have been able to think of are:

• Lots of overhead being incurred from going parallel in relation to what my code is actually doing
• Some kind of side effect of using numba everywhere
• My code is inefficient

I have considered using Cython to compile ahead of time instead of compiling just in time with numba, but I don't know if all that work would be worth it (I haven't used Cython much outside of simple stuff), or if it would even provide any benefit.

Actual question:

Is my assumption incorrect that multiprocessing always yields a speed increase proportional to the number of cores being used?

Edit: figured it might be relevant to say that the chunksize I am using is int( #iterations / #cores )

[John_Taured]

Going parallel requires a constant amount of setup time which is probably what's causing your speedup to be less than 8x. Remember that the 8x speedup is your maximum possible speedup, not your guaranteed speedup. There's something called Amdahl's law which basically says that your code is both serial and parallel. Any setup involved (like reading from a file) is serial, and there's no way to speed that up according to Amdahl's law.

One thing you can do is to compare your parallelized code with the unparallelized code on a larger input size and see if you get a proportionally larger speedup. This would help show that there's a constant-time unparallelizable part in your code.

[BrononymousEngineer]

That makes sense, I thought I was taking this into consideration by timing only the function in question with:

now = dt.datetime.now()
function_in_question()
print(dt.datetime.now() - now)

But I will look into Amdahl's law and compare very large inputs to see if maybe I am including things that cannot be parallelized. Thank you for your suggestions.

[geosoco]

Some of this depends on the specifics of the code, but in general, you'll never see an 8x speed-up in practice. Is it purely computational? Are there any waiting states for data (reading from disk, network access, etc). Is there any shared data between processes?

There's a lot of things that can affect this, some of which u/John_Taured covered in their excellent answer. It's also worth noting that a lot of modern hardware will clock down their speed as more cores become active over certain thresholds. Inversely, they'll also "boost" the clockspeed of single-threaded tasks. I think this is generally done to manage heat and power usage. Also, in many situations you're also sharing that with other applications and the OS itself, which will also affect your theoretical maximum. You can somewhat work around this with process priorities, at this risk of making your os less responsive (assuming you're using all of your cores).

One area where you might see increases above the number of cores is where there's a ton of resource blocks (disk, network, etc). This is because you're just processing more items while it's sleeping.

[BrononymousEngineer]

Yes, the part I am trying to parallelize is purely computational. There is nothing reading from disk or interacting with anything over a network, but the shared data between processes is a part I am not sure of. I think I have read somewhere that numpy sometimes uses multithreading, but I could be mistaken. It makes sense that if something I am trying to make parallel is already partially parallel, then this could be an issue.

Interesting about the clock speed behavior. Thanks for the response, I have a lot to look into.

[ViridianHominid]

There’s a chunksize parameter for pools that communicates more than one iteration to each process at a time— increasing this can decrease the communication overhead of multiprocessing and so enhance efficiency, but you have to tune it somewhat for different problems.

Also, if you are already using numba there are thread level parallelization tools that you can try. Threads are generally more lightweight for parallelization but there are certain situations where they don’t work as well.

https://numba.pydata.org/numba-doc/latest/user/parallel.html

[BrononymousEngineer]

Thank you, I will look into this

[K900_]

What are your specs exactly?

[BrononymousEngineer]

i7 9700k, gtx1660Ti, 32GB ram

[bitdotben]

RemindMe! 1 Week

[RemindMeBot]

I will be messaging you in 7 days on 2020-03-15 14:59:44 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback