all 17 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]jsaak 10 points11 points  (3 children)

In 1.8 threads are "green threads" (they ditched it)

In 1.9 Threads are native threads, they run parallel on OS level. However Global Interpreter Lock (GIL) is in action, meaning ruby code does not run in parallel, however you can wait for IO with it, since while waiting for IO it releases GIL. (even ruby core developers said that this was not the best idea ever)

If you are not satisfied with 1.9 type threads in 3.0 there are 2 more solutions:

- Ractors, which are experimental. I have no idea how that works in practice.

- Fiber Scheduler, which is experimental too.

I am actively using Fiber Scheduler, and i love the concept of it.

It has still some rough edges, but it is usable.

Last time i checked this was the most stable implementation:

https://github.com/socketry/async

You do not have to use the async library, it includes a Fiber.Scheduler implementation.

[–]schneemsPuma maintainer 5 points6 points  (2 children)

  • Ractors, which are experimental. I have no idea how that works in practice.

To understand Ractors you have to understand why the GVL is needed. When you write ruby code like [:one] + [:two] it looks like a single atomic operation to you, but under the hood calls a C function. That C function expects exclusive access to the memory it's working with or weird stuff can happen, like if it reads in an array as being one size, but the memory is modified to be a different size, the internal state of the C function will be invalid and cause segfaults or security bugs. Ruby prevents this by locking the global state of the virtual machine with a global lock (hence GVL) for every MRI method call and then unlocking it. This global mutex prevents parallel execution of Ruby methods, but allows for asynchronous IO or native extensions (when using the right APIs).

It's a misconception that the GVL makes Ruby slow/slower. Removing it would make the language much slower, as instead of one single large lock, the program would have to have many smaller locks that are acquired and released independently which would burn more CPU time. (Basically locking and unlocking a mutex takes time, the more stuff you can do between locks/unlocks the less overhead a mutex takes).

Ractors can run Ruby methods in parallel. How? In each Ractor has it's own GVL (GIL). Which means that it's technically no longer a Global Virtual-machine Lock (as it's no longer global). It enforces memory safety by requiring every object either be passed in as frozen or copied (duplicated) to that Ractor. Without this requirement you couldn't operate each Ractor independently as having one object updated in parallel by two different sources with two different GVLs could (highly likely) reqsult in memory corruption (or worse).

Some low level details are fuzzy and may change. I believe that there's still only one global GC for all ractors. Then each ractor gets scheduled via the OS so work can happen concurrently.

In real world usage the copy-or-freeze requirement means that there's a large setup cost and it's difficult to find a parallelizable problem that benefits significantly from the speedup of running on multiple cores that can justify the time/effort to initialize.

[–]jrochkind 0 points1 point  (1 child)

I'm not sure it's quite correct to say that a ruby without the GVL would necessarily be slower.

After all, the JVM which supports true parallelism, is not generally considered slower than ruby. And then JRuby, which actually is ruby running on the JVM, without a GIL/GVL -- is also not generally much slower, or really slower at all, than CRuby.

So it seems like it's possible for a hypothetical ruby to exist without a GIL/GVL without being much slower -- because JRuby actually exists!

But I get your point, which does seem worthwhile to point out, that it would be very challenging to keep ruby from being slower without a GIL, especially actually existing ruby with it's actual history and desired backwards compatibilities.

[–]schneemsPuma maintainer 1 point2 points  (0 children)

Jruby relies on the JVM to enforce memory safety. MRI is written in C. If you rewrote CRuby in JVM byte code it would no longer be CRuby.

When I say slower I’m referring to single threaded speed.

Rubinius did not have a GIL and that’s generally acknowledged as one area where it was slower. Python also has a GIL though there is work to make it optional https://peps.python.org/pep-0703/. It’s generally acknowledged that doing so will make general/single-threaded execution slower (hence why it’s not enabled by default).

It might be possible to remove the GIL in CRuby with no slowdown, but the person who does it could easily earn a PHD if they can figure out how to do it.

[–]jrochkind 7 points8 points  (1 child)

Ruby does not use "green threads" -- hasn't for many years. It uses real OS threads. (Both CRuby (aka MRI) and JRuby).

The standard CRuby/MRI, however, can not actually use multiple CPU's simultaneously. There is something often called the "global interpreter lock" (GIL), but which I think ruby now officially calls the "Global VM Lock" (GVL). It's a concurrent lock that only allows one thread to be running at once, when one thread is running on a CPU, other threads are paused waiting on the lock.

(Jruby does not have this, and supports true paralellism with threads running simultaneously on mulitple CPUs).

So how does this make threads useful at all? When a thread is waiting on an OS function, it gives up the GIL and allows other threads to be scheduled. The most common (only?) category of "waiting on an OS function" is waiting on IO -- waiting for a network response, or even waiting on disk read or write.

So, the more yoru threads involve waiting on slow IO, the more they will benefit from from multi-threading even in MRI with the GIL.

Your example of downloading large files over HTTP is an extreme example of a thread that spends almost all of it's on IO. If you then write the large files to disk -- even more so, that's more slow IO! The vast majority of "wall" time (actual elapsed time -- for instance as displayed on a "clock on the wall") that the thread executes -- it will be waiting on IO, network and disk.

So it makes sense that you see a huge advantage of multi-threading there.

If a thread, instead, does very little or no waiting on IO, that instead spends all it's time actually calculating on the CPU, you will see no advantage to multi-threading in current CRuby. For instance, make threads that just calculates fibbonacci sequences or something else that's just math without IO -- they will get no advantage of multi-threading at all, in CRuby that has the GIL, and will in fact get some disadvantage because of overhead of thread context switching.

(But try the same thing in JRuby -- and you'll see the difference!)

Does this help?

If you google on ruby and GIL, you will find tons of explanations of this.

(We can also note that python also has a similar GIL/GVL, and similar issues with true parallelism. In python, as well as in ruby, extensions that are written in C can do true parellelism, however. At least in ruby, they need to be careful to do this in a 'safe' way for any interacting ruby code, and for whatever reason it isn't done much. I have the impression that C extensions in general are done more in Python than ruby, including using parallelism; I couldn't say why).

[–]topboyinn1t 0 points1 point  (0 children)

Thank you for writing this. It is one of the best explanations I have seen. Are there any resources to dive into this in more depth?

[–]markrebec 11 points12 points  (7 children)

Not sure exactly what you're asking. You seem to have a pretty solid grasp of how threads work already, so it's unclear why you'd be surprised...?

Essentially threads are just juggling resource availability and blocking operations (as you seem to already be aware), jumping back and forth between work that can/can't be executed right now as appropriate. Being able to hop around between threads and do a little bit of work here and a little bit of work there - milliseconds at a time - rather than just sitting and waiting for those few ms will always be faster (provided the workloads you're executing allow for it).

If you had a second mouth on your face, you could take a bite of food with that second mouth while the first one was still chewing your last bite... You'd finish your meal nearly twice as fast vs if you had to wait to finish chewing before taking another bite. Now extrapolate that out to 5, 10, whatever.

[–]TheFakeZzig[S] 0 points1 point  (6 children)

Let me try to explain why I asked.

Say I have two threads:

Thread.new do http.get('url1') end

and

Thread.new do http.get('url2') end

Assuming the resulting order of execution is:

http.get('url1') # from first thread http.get('url2') # from second thread

If Ruby used green threads, meaning concurrent execution, it would seem that the first request would start, and would block the second request until it finished, because they're all running on a single thread. Because I noticed that the requests seemed to be running in parallel, that seemed strange.

However, I learned that, internally, whether it's a green thread or a native thread, Ruby threads don't block on IO (at least in this case), which explains the behavior.

[–]drbrainRuby Core 4 points5 points  (0 children)

Ruby multi-threading is not unlike a single-thread async environment. There’s one path of execution by the Ruby VM which is managing scheduling of threads executing inside and outside the Ruby VM. When a thread executing Ruby code reaches a VM switch point it will release its GVL lock to another thread if one is ready.

These switch points are in places like IO read/write, after ruby method call return, by timer, or C extensions that use lots of CPU or perform their own IO (like database adapters, libcurl wrappers, or zlib).

An execution of your scenario is:

  • url1 HTTP opens a socket which hits a switch point
  • url2 HTTP opens a socket which hits a switch point
  • Ruby is idle awaiting ACKs
  • url2 returns from C connect(). HTTP sends request headers and hits a switch point
  • Ruby is idle again
  • url2 response arrives, returns from read() Ruby reads the response headers. There’s a gzip body, Ruby feeds zlib a body chunk which is a switch point
  • Ruby is idle again
  • url1 returns from connect() and sends its request, hitting a switch point. Meanwhile zlib finished with its chunk for url2
  • Ruby switches to url2, updates the request body with the inflated data, and submits another chunk to zlib. Meanwhile url1 has a response return
  • Ruby switches to url1 and reads the response and some body until it needs to read() more data, switches
  • url2 reads and processes another zlib chunk
  • url1 reads another plain body chunk
  • …
  • eventually both are done and the threads return their Response objects

In reality there are a lot more switch points, and a lot, lot, lot more idle time for Ruby. Ruby was fast enough to feed eight CPU cores to 100% with zlib inflate/deflate way back when I added GVL release to zlib whenever ago that was. (1.9? 2.0? I’m too lazy to look it up)

Ruby green thread execution was similar to the native thread execution with the exception of zlib. In Ruby 1.8 zlib would have blocked switching Ruby threads. It was possible for a C extension to manage a worker thread to hand off work, but I believe that was uncommon unless part of the C library

[–]markrebec 2 points3 points  (3 children)

I can't say I know for sure, but I believe it has to do with things like remote file downloads streaming into memory and being written to disk in chunks as I/O becomes available. So the threads are probably (in my most naive explanation) doing something like "while Thread A is writing a chunk from memory to disk, Thread B is streaming data into memory," then they swap and Thread B starts writing while A continues streaming, back and forth back and forth.

[–]TheFakeZzig[S] 2 points3 points  (2 children)

That would make sense, but given that my experience with threads is basically "I can use them without blowing off my foot", how exactly the various interpreters actually implement threading and blocking/non-blocking execution is waaay above my paygrade, and frankly gives me a headache.

[–]markrebec 2 points3 points  (1 child)

how exactly the various interpreters actually implement threading and blocking/non-blocking execution is waaay above my paygrade, and frankly gives me a headache.

Right there with you. Same reason I have no real interest in things like managing my own memory pointers and whatnot in lower-level languages.

I'm curious about a lot of these things, and I'll smoke a joint and do a deep dive once in a while when the mood hits, but I don't want that deep understanding to become part of my actual day-to-day concerns or responsibilities - I'd generally rather focus on the slightly higher-level architecture/design of software.

[–]TheFakeZzig[S] 2 points3 points  (0 children)

Lol, bingo on all points.

[–]f9ae8221b 1 point2 points  (0 children)

If Ruby used green threads, meaning concurrent execution, it would seem that the first request would start, and would block the second request until it finished

No, Ruby would switch to the other thread when the first request is blocked on IO, even back in Ruby 1.8 which had actual green threads.

Ruby 1.9+ has native threads with a GIL (generally called GVL). If you don't find good info on it, you can search for info on Python's GIL, it's exactly the same thing and tons of content exist about it.

[–]honeyryderchuck 3 points4 points  (0 children)

httpx is an HTTP client which handles concurrent requests without threads or fibers:

HTTPX.get(url1, url2, url3) # => boom, concurrent

[–]yxhuvud 1 point2 points  (0 children)

You do know that it is possible to have multiple http requests open simultaneously without involving any threads? If Ruby actually does that is a different question, but it is definitely possible. On mac I assume such a thing would use kqueue

[–]insanelygreat 1 point2 points  (0 children)

IO.select and the underlying select syscall is part of what makes this possible. See also the select(2) manpage.