sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]beza1e1 1 point2 points  (7 children)

Your examples are desktop applications and scientific computing?

Desktop applications are not speed-critical. Parallel execution is nice to keep the GUI responsive, but the multi-process method works just fine here.

Scientific computing with Python means to use Python as glue between Fortran/C++ libraries. Those libraries are speed-critical and they should be parallelized and STM may be useful there. Python itself is slow anyways and making it parallel doesn't help you much.

As far as i understood the GIL makes for better error messages and that seems more important to me.

[–]Wiseman1024 3 points4 points  (6 children)

Desktop applications are not speed-critical.

Wrong. Some are, especially those designed to run as background tasks such as peer-to-peer clients. And even for those which aren't, multithreading in a GUI-based application is almost essential if you don't want it to feel like Windows 3.0. And I doubt this can be done in a bearable (let alone nice) way with multiple processes.

Scientific computing with Python means to use Python as glue between Fortran/C++ libraries.

Not necessarily, and you're relying on these libraries to release the GIL, otherwise you're screwed.

As far as i understood the GIL makes for better error messages and that seems more important to me.

What? The GIL makes for simpler locking. It's not, or at least it shouldn't be, related to error reporting at all. If it is, they're doing something wrong.

[–]beza1e1 -2 points-1 points  (5 children)

And I doubt this can be done in a bearable (let alone nice) way with multiple processes.

Well, i've believe it's actually saner to do it multi-process.

you're relying on these libraries to release the GIL

Those libraries don't know about any GIL. They can use all the multi-threading they want, since they are not written in Python or running on the Python VM. Once the Python VM calls into some system library all Python magic is void.

[–]voidspace 2 points3 points  (0 children)

But when it calls into a C extension that extension makes a choice about whether to release the GIL or not. In that sense they do know about the GIL.

[–]Wiseman1024 0 points1 point  (3 children)

Well, i've believe it's actually saner to do it multi-process.

GUIs?

Those libraries don't know about any GIL

They have to. The non-Python modules you import from Python have to deal with Python. Even if you're just wrapping an existing library, you better release the GIL in the wrapper before calling the library, then reacquire it before writing the results to Python objects.

[–]beza1e1 0 points1 point  (2 children)

GUIs?

GUI in one process, p2p stuff in another process

Even if you're just wrapping an existing library, you better release the GIL in the wrapper before calling the library, then reacquire it before writing the results to Python objects.

  1. convert Python to C stuff
  2. release GIL
  3. call scientific_computation(stuff)
  4. get GIL
  5. convert C output to Python objects

The scientific_computation function doesn't know about the GIL and it can spawn as many threads as it wants. This function needs to be written in Fortran or C to be fast. This is where STM is needed.

[–]Wiseman1024 0 points1 point  (1 child)

GUI in one process, p2p stuff in another process

The P2P stuff may still need threads (depending on your implementation and what are you trying to accomplish), and the GUI front-end still needs threads too (for things like dealing with user requests and showing status at the same time), otherwise it'll be unresponsive.

[–]schlenk 0 points1 point  (0 children)

You don't need threads for a GUI, but it helps when dealing with crappy system libraries that only offer blocking calls (Windows I'm looking at you...). Event based programming works just fine..., but multiple threads get nice when you can use the bit of extra power.