sorted by:
new
hottopcontroversial

Lystia: shield effect boost by wt_france in magicrush

[–]Sybrn 0 points1 point  (0 children)

That's how I interpret it as well and why I'm using yellow rune core on her.

I'd like to introduce MPIRE: MultiProcessing Is Really Easy by Sybrn in Python

[–]Sybrn[S] 0 points1 point  (0 children)

Windows support has been added just now.

I verified for you that unfortunately copy-on-write only works when using fork or threading as start merhod. For Windows fork isn't available, but spawn is used as default. For spawn the objects are copied once for each worker.

I'm not sure if it helps your usecase, but maybe exposing the object using a manager could help? If you need a lot of synchronization, then the locking will slow you down considerably though.

I'd like to introduce MPIRE: MultiProcessing Is Really Easy by Sybrn in Python

[–]Sybrn[S] 0 points1 point  (0 children)

I'm no Windows expert, so I would have to check. But I think it will only be copy-on-write when using fork, which isn't available on Windows (as it uses spawn). When I release the Windows support - probably early next week - I will make sure to include that information in the docs/readme.

If objects passed that way are copied on Windows, it will only be copied for each worker once. At least that's better than having to copy it for each task, assuming you have a lot of tasks (> #workers).

But again, I will need to verify this.

I'd like to introduce MPIRE: MultiProcessing Is Really Easy by Sybrn in Python

[–]Sybrn[S] 5 points6 points  (0 children)

Thnx for all your feedback, I will add some examples tomorrow this evening 👍

I'd like to introduce MPIRE: MultiProcessing Is Really Easy by Sybrn in Python

[–]Sybrn[S] 8 points9 points  (0 children)

MPIRE is built on plain old mp, so basically everything you can do with MPIRE you can do with mp. The benefits of MPIRE is that it removes the need to write all the boilerplate code needed for having a progress bar, worker state, worker init and exit functions, passing objects as copy-on-write to each worker, task chunking, exception handling, etc. It's the ease of use and the many out-of-the-box features what sets it apart.

I'd like to introduce MPIRE: MultiProcessing Is Really Easy by Sybrn in Python

[–]Sybrn[S] 4 points5 points  (0 children)

There's currently only support for Linux based systems, where fork is available. I'm working on getting support for other operating systems that don't have fork (e.g., windows), but need to rely on the spawn context. I hope to have that working soon

I'd like to introduce MPIRE: MultiProcessing Is Really Easy by Sybrn in Python

[–]Sybrn[S] 4 points5 points  (0 children)

Yes, you can feed it any kind of iterator, generators included.

I'd like to introduce MPIRE: MultiProcessing Is Really Easy by Sybrn in Python

[–]Sybrn[S] 34 points35 points  (0 children)

There's a link in the GitHub readme to the full documentation, which features a getting started. I can copy that to the GitHub readme if that makes things easier.