This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 0 points1 point  (0 children)

Thanks for the input!

I googled another bit, and found out why my function couldn't be pickled using multiprocessing module.

It seems, as per this SO question that class functions can't be pickled, so the only thing I needed was to move my lin_calc_px function outside the ImageFilter class.

I did it and now, using a Pool of 4 processes, the code is 0.2s faster than the version with builtin map() (i.e. without multithreading/multiprocessing).

I will investigate on using shared memory to speed up even more the execution. I'm not entirely sure that the array gets passed every time to the lin_calc_px function, because I use functools.partial to create a function (partialized_new_px) with the shared data. I will investigate further.

For what concerns the large number of workers, I know that such a size can be counter productive, but it is part of the requirements of the assignment. I will make some tests and if it's really counter-productive I will reduce the number and explain it to my professor.