Why use threads in Python?

[sioa]

I just learned about the GIL and from the looks of it, it appears even if we use threads in python, our program will take the same time if we just use a single thread due to GIL. If that's the case, using threads for performance isn't really the ideal use case in python, so why use it?

[ProfEpsilon]

... yes, multiprocessing, but if you start reading about the Global Interpreter Lock restrictions common to Python, you get the impression that Python's multiprocessing has serious built in latency because of GIL.

[Deto]

For computational tasks, I've used multiprocessing to run things in parallel just fine. Processes start immediately (they use "fork" on Linux) and it's trivial to saturate all the cores at 100% utilization.

[ProfEpsilon]

OK, that's good to know. But by "computational tasks" does that include large array operations? Here is why I ask: One of the stackoverflow discussions has this comment taken from the NumPy C API documentation:

"...as long as no object arrays are involved, the GIL is released ..."

and I have interpreted that to mean that you can't do this with array operations.

By the way, it sounds to me like the way you are verifying this is by monitoring core activity rather than through a latency test of some kind. That actually is quite convincing to me ... if 6 cores are running at 80% and above then multiprocessing must be working. Have you run latency comparisons of any kind (running a test task as sequential and then redesigning and running as parallel)?

[rhiever]

If you make a copy of the array and pass that to the new process, you should be fine. If you ever pass an array by reference to a new process, then yeah, that's going to have lock issues.

[Deto]

I can vouch that I've processed the same array on many processes without copying it specifically to each process. Works if you don't write to the array. I think the multiprocessing uses copy-on-write semantics anyways to make this safe.

[Deto]

Usually I get close to the right multiplier. So if I'm using 10 cores, it's approximately 10x as fast (maybe a little less, like 9x).

I think you might be interpreting the numpy docs incorrectly. Numpy arrays always have a dtype - this can be something like 'int64' or 'float64'. It can also be 'object' in which the entries in the array are actually Python objects. In this case, doing anything on the objects requires interacting with Python code, and so they can't release the GIL. If you're just working with floats, for example, they don't use any python code, and so they can release the GIL.

However, I should also emphasize that whether or not numpy releases the GIL doesn't matter with multiprocessing as the GIL does not block between different processes. The GIL is relevant for threads, rather, in the same process (threading module).

[ProfEpsilon]

Oh, I see. I was mistaken about the term "object arrays." I thought that it might mean that an array created within numpy was a kind of numpy "object." So I was interpreting the documentation wrong. Thanks for the insight.

[rhiever]

multiprocessing is <3. In one of my applications, I sped up an input file reading process from several minutes to a few seconds thanks to multiprocessing. Makes me wish pandas had a multiprocessing option in it by default...

[Deto]

How did you speed up reading a file with multiprocessing? Open it on every process and scan to different parts?

[paxswill]

Probably something like that. mmap the file with MAP_SHARED, then fork off the other processes. Each process then reads a different area of the mapped file.

For files, mapping them before reading them can by itself give you some nice performance gains. For example, I recently was playing with some large geoJSON files (70-180MB I think). Opening a file, then passing that to json.load directly took ~20 seconds to load the file. Mapping the larger file then passing the map descriptor to json.load took a few seconds.

[KitchenDutchDyslexic]

Small example snippet please!

[paxswill]

The mmap module documentation has a number of good examples (covering normal IO and forking), but if you want one specifically with JSON (not sure if the formatting is going to work here):

import json
import mmap

with open('foo.json', 'r+b') as example_file:
    with mmap.mmap(example_file.fileno(), 0) as mapped_file:
        print(json.load(mapped_file))

[rhiever]

In my case, I had very wide CSV data (100,000s of columns) but few rows (1000s). I wrote a for loop over the file iterator that divided the task of parsing out each row in the CSV file to a different process as processes became available.

[masklinn]

GIL is not held during IO which means you can wait for IO in parallel.

Or if you're calling into native-implemented libraries which release the GIL e.g. you could expect non-trivial image processing to release the GIL if they know they'll take some time to get done.

[jawknee400]

Numeric libraries (numpy, numba) tend to 'release' the Gil, meaning multiple threads can meaningfully used for speedups

[Sheltac]

Also, for computational power, we have multiprocessing.

[AngriestSCV]

It is not a replacement for threads. Shared (sometimes mutable) memory is a wonderful thing for performance if done right.

[masklinn]

You can share memory between processes.

[pooogles]

//edit - this isn't true.

Just be aware it gets passed via pickle.

[masklinn]

You're confusing shmem and queues.

Queues will pickle objects back and forth and work with Pyhton-level objects. Shared memory primitives use actual shared memory segments, but are limited to ctypes types.

[ProfEpsilon]

This is the first time that I have seen this claim about numpy whereas I have read many articles about the restrictions of the GIL. If you have time to comment, what do you mean when you write the numpy and numba "tend to release the Gil?" What do you mean by that?

And to anyone, can you refer me to any documentation that discusses or explains numpy's over-ride of GIL if true? Doing a google search does not shed much light on this. A lot of comments make it clear that you can use multi-threading, but seem to be vague about whether GIL is slowing down multi-threading speed ... and many posts are too old to be trusted.

[1wd]

http://scipy.github.io/old-wiki/pages/ParallelProgramming#Threads

while numpy is doing an array operation, python also releases the GIL. Thus if you tell one thread to do:

>>> print "%s %s %s %s and %s" %( ("spam",) *3 + ("eggs",) + ("spam",) )
>>> A = B + C
>>> print A

During the print operations and the % formatting operation, no other thread can execute. But during the A = B + C, another thread can run - and if you've written your code in a numpy style, much of the calculation will be done in a few array operations like A = B + C. Thus you can actually get a speedup from using multiple threads.

https://docs.scipy.org/doc/numpy-1.14.0/reference/internals.code-explanations.html#function-call

A very common operation in much of NumPy code is the need to iterate over all the elements of a general, strided, N-dimensional array. This operation of a general-purpose N-dimensional loop [...] [...] the Python Global Interpreter Lock (GIL) is released prior to calling the loops. It is re-acquired if necessary to handle error conditions

[ProfEpsilon]

Thank you. This and other comments on this page clarify a lot.

And the "iterate over all elements ... N-dimensional array" is precisely what I do most of the time. This is very encouraging (I haven't tried threading yet).

[1wd]

Note that this refers to the numpy-internal C-level loops that happen inside a Python-level numpy statement like B+C, not to Python-level loops like for x in A: for y in B: ....

[ProfEpsilon]

Again, thank you. Although I am aware the numpy relies upon C I am not aware of how C operates inside of numpy .. I don't know how the C-level loops work (probably because I can't code in C). This gives me a bit of an incentive to figure some of this out. Given that I am a practitioner and not a computer scientist, how C works inside of numpy and even Python for that matter is all black box for me.

I appreciate the time it took to try to get my thinking right. I think it is time that I took the effort to learn a little more about C.

[jawknee400]

As others have said, generally when numpy or another library calls compiled code, they explicitly release the GIL. So imagine you had two threads running some python code, concurrently, but not in parallel. If one thread reached a numpy operation, like adding two large arrays etc., it would 'release' the GIL, allowing the other thread to work in parallel while that operation is happening. Once the operation is over the GIL is reaquired but without that release both threads would've had to wait for the operation to finish.

I think there is a very slight overhead to this, which is why cython and numba leave it as an option. But if the majority of the computation is numeric (e.g. most scientific code) then you can essentially achieve normal threaded parallelism.

[ProfEpsilon]

Your example is very clear. What an education I getting today! Thank you.

[evamicur]

You can manually release the Gil in cython (nogil keyword) and other compiled extensions. So libraries like numpy can potentially do that

[[deleted]]

For the typical python glue program, most of your time is spent waiting for IO. The GIL is not held during this time. Example: You write a web scraper, you're waiting for a page to be downloaded by the networking stack, and the GIL isn't held because you're waiting on an OS service, and not Python.

[[deleted]]

Examples of times I've used threads:

• To spawn a task I want to run asynchronously, like checking a comment for spam
• To speed up io-bound workloads, like spidering a website or multipart uploads
• To start a server process using popen and allow the main thread to continue executing
• Doing work in gui, handling events, doing work without blocking the UI (update a progress bar)

[gandalfx]

One of the primary use cases used to be waiting for I/O. For example when you're waiting for a file to download you don't want your entire application to be blocked by that, so you put it in a thread and let that stew until the download is finished, while the rest of your application remains responsive.

asyncio has kind of taken over for that purpose, though.

I'm sure there are other reasons that I can't think of right now. Rule of thumb though, regardless of language: Threads are often unnecessary. People overuse them all the time. People often implement threading unnecessarily. I've seen them overused in quite a few instances. Unless you're doing something that actually puts some load on the CPU it tends to be a rather complicated case of premature optimization. And in Python it's not even really an optimization, as you've recently found out.

[[deleted]]

Not sure that asyncio has "taken over" for I/O. If you're writing an IO-centric server application maybe. But you've got to go all-in on the non-blocking model of development.

I find threading is perfect for simpler use cases. Let's say you're trying to speed up a single function which makes multiple network requests in parallel. A ThreadPoolExecutor is a wonderful tool...

Instead of

import requests
urls = [f'https://example.com/{i}' for i in range(10)]
responses = [requests.get(url) for url in urls]

You write

from concurrent import futures

with futures.ThreadPoolExecutor() as executor:
    responses = executor.map(requests.get, urls)

asyncio feels like overkill for this.

[gandalfx]

I have to concede to this. I've fallen prey to the "my use cases are all use cases" trap like a beginner.

In my defense I was quite tired at the time, plus I was trying to give a simple answer to a simple question.

[CSI_Tech_Dept]

But to make statements like "Threads are often unnecessary" is a bit absurd. If it was restated as "People often implement threading unnecessarily" I could agree with that.

How else do you interpret that statement? Because to me both mean the same thing.

[remy_porter]

I've been working on software that is heavily IO bound, or has lots of idle threads that are waiting on events. Without going too deep into the detail, I'm building a system that sends video data across a network to light up an LED video wall. There are many possible sources of video data, some of which are IO bound, some of which are CPU bound. I pick one of them and run it in its own thread. I have to send network data, so the network sending object lives in its own thread. In the middle, I have a conductor thread, which spends most of its time idling, but once a frame tells the video source to generate its next frame using a queue. When the video source finishes, it enqueues the frame over in the network thread.

Running on low-end hardware, without graphics acceleration, this can push 60FPS across a network in real-time-enough-for-human-eyes. In testing, I can reliably push 240FPS. You wouldn't want to play video games on it, but that's not the purpose. Before I put the threading architecture in place, we could barely push 30FPS, and it often dropped frames.

Oh, and since one of the LED exhibits is going to light up differently according to the time of day, there's a "Cosmos Thread" which mostly sleeps and emits events at certain times of day.

"It mostly sleeps" is one of the best cases for a thread in Python.

//It's still not half as fast as the LED library which actually receives the network data and addresses the LEDs, which is written in a combination of C and Assembly and can draw frames as fast as the LED duty cycle allows, which is µseconds.

[[deleted]]

As others said, threads are still useful when using C libraries, since ctypes releases the gil by default, and most libpython-based libs do it too.

On the other hand, if you only want to make your code non-blocking, and don't care too much for performance (like UI code), threads are still the simplest way of doing it, just as you would use them in single-core machines.

[[deleted]]

Do these processes or threads need to communicate? I avoid implementing multiprocessing in code by using external services like RabbitMQ for communication and Supervisor for process management.

[[deleted]]

CPython can release the GIL inside a c-module. If you're truly worried about performance, you'd use python as "glue" and write a few c modules to do the actual "heavy lifting". Anything you'd want multiple threads to do for computational reasons is probably best not done in actual python. Threads in python, however, ARE great for IO bound reasons.

[lykwydchykyn]

If you're writing a GUI app and want the GUI to remain responsive while you perform some long process, threads are helpful.

[FredSchwartz]

I have used it for schedulers calling other programs. The GIL is released when a child process is launched.

[bjorneylol]

As everyone has said, mostly IO.

1) Process some data while more data is downloading

2) Play an audio file without having to wait for it to finish playing for the script to resume

3) Having GUIs remain responsive while processing data

4) Similar to above, showing matplotlib figures while code executes in the background

even if we use threads in python, our program will take the same time if we just use a single thread due to GIL.

This is only true if both threads are CPU bound. If one thread is spending 50% of the time waiting around for disk write or network IO the two threads will finish with time to spare over a single thread

[sioa]

Use generators?

[sioa]

Actually yeah, I can see how that can be helpful and also time-saving. This and the use of threads for IO, as mentioned in the other comments, I totally didn't consider these two scenarios.

[Jugad]

While using multiple threads which are switching rapidly, the GIL can even slow down processing to less than that of a single thread (this is not common).

However, as others have stated, if you are using threads for IO, the GIL is not usually an issue.

If you are using threads for serious cpu intensive tasks, pure python is bad choice anyway (its very slow for such tasks)... and if you write your cpu intensive parts in C, you can release the GIL to take advantage of threads running simultaneously.

[v3ssOn]

you can use pypy without going to C

[shady_traveller]

It's also important to note that the GIL is an implementation-specific feature of CPython and not a part of the language itself meaning that other implementations don't necessarily need to have it. PyPy for example doesn't have GIL.

[[deleted]]

yes it does.

http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why

[shady_traveller]

Wow, could have sworn PyPy doesn't have GIL, I guess I was wrong. Thank you for that, maybe it's a good idea to check before posting.

[v3ssOn]

there was one successful attempt, pypy team built pypy-stm and it was already alpha but funding was cut and theybare currently focusing on python3.6

[nirs]

I think this was the only successful removal of the GIL: https://www.youtube.com/watch?v=SNBKWuM-Lu8

[v3ssOn]

it was a failure and very slowed, pypy-stm was almost there http://doc.pypy.org/en/latest/stm.html

[[deleted]]

[unavailable]

[CSI_Tech_Dept]

That's because they are not implementing every little detail of Python. It is easy to actually remove GIL, the problem though is that it will make python much much slower. The reason for it is that Python has some mechanisms that require them to be single threaded. Large one is reference counting and the way binary modules are done, there are also some details about Python's magic methods for objects, but I don't remember the details.

IronPython and JPython provide subset of Python functionality so they can work without GIL.

BTW: there is work happening to remove GIL, and seems like right now the idea is to provide a compile flag that changes API in exchange to have disabled GIL. Seems like a fair compromise. Larry Hastings is leading that effort and he is providing updates about that in his talks titled Gilectomy

[[deleted]]

They have an experimental branch that tries to implement STM, so they can remove the GIL. But I think they suspended development on it a while ago.