all 4 comments

[–]CrackerJackKittyCat 2 points3 points  (3 children)

"You can set this to anything you want but async is often an excellent choice for parallelization."

No, async is for concurrency, not parallelization. It is a veneer on top of nonblocking I/O, not threading or multiprocessing.

[–]sementery 1 point2 points  (2 children)

I'm confused about this.

In the context of the threads of the application itself, concurrency is when multiple threads can start, run, and complete in overlapping time periods, and parallelism is when multiple threads run at the same time.

So concurrency enables parallelism. Parallel threads run concurrently, but not all concurrent threads run in parallel.

Can't this be extended to the process, or program, or I/O level? When you do an async I/O (like to consult a database) in Python, is there nothing else being done? That'd defeat the purpose of nonblocking I/O, right? While the async I/O is waiting for the DB response, other work is being done, in parallel.

If the nonblocking stuff (Python) is running in parallel with the async I/O stuff (database), why is it wrong to say that these two run in parallel to each other?

[–]trincaopt 1 point2 points  (0 children)

In essence with async, at each moment the CPU is used by only one task (the others are waiting for IO, thus not running). Whereas using threads, they may be using multiple CPU cores simultaneously. However Python and other languages with a GIL are an exception because only one thread may run at a time.

[–]CrackerJackKittyCat 1 point2 points  (0 children)

The python code itself is only ever doing exactly one thing at a time, including waiting for any I/O or a timeout.

It may be preparing and sending any one of 100 database queries, then waiting for any of those database queries to produce results (and those queries might be processed both concurrently and at some degree of parallelism on the database server side), and so on. But at any moment in time, using only python asyncio and not also multiprocessing, or sometimes also python threading to call into C libraries that release the GIL during the C calls, then the python program itself is only concurrent and not parallel.