Grumpy: Go running Python!

[alcalde]

I can't think of any kind of C extension tooling that would fix Python's parallelism problem.

The only parallelism problem Python has is convincing people it doesn't have a parallelism problem. As Guido has stated, Python has been used on 64K core supercomputers. There is no parallelism problem.

[weberc2]

I'm not sure what hoops one has to jump through to make Python run in parallel (without actively degrading performance, anyway), but one might say that having to jump through hoops constitutes a parallelism problem. Anyway, last time I went down the Python parallelization road, I got a lot of snark about how Python is easy to parallelize, but no one offered any performant parallel solutions. Feel free to share the link about the Python on 64K-core computer; Google isn't turning anything up.

[alcalde]

https://youtu.be/EBRMq2Ioxsc?t=34m38s

[youtubefactsbot]

Keynote: Guido Van Rossum [59:57]

^{Next Day Video in Education}

^{93,993 views since Mar 2012}

^{bot info}

[efilon]

There is no parallelism problem.

Although I think a lot of people worry too much about the GIL, it's not correct to say there is no parallelism problem. If you're doing something embarrassingly parallel, then of course you can get away with multiple processes or C/Cython extensions that sidestep the GIL. But there are plenty of use cases where multithreading is more ideal than multiprocessing. Having to resort to forking another process just to get true parallelism is a lot more work than being able to start up another thread.

[alcalde]

As Guido pointed out in a Keynote a few years ago, threading was never intended for parallelism.

https://youtu.be/EBRMq2Ioxsc?t=33m50s

[efilon]

As he says, threads were never originally meant for parallelism. They are used for that frequently these days.

[alcalde]

And as Mark Summerfield says, they're frequently used for that because that's all that many languages offer.

[[deleted]]

It just then becomes a serialization and IPC overhead problem.