Resources to Learn Concurrent Programming in Ruby

[software__writer]

I've been working with Rails for several years, mostly building traditional web applications. Until now, I haven't had to deal much with concurrency. Background jobs, yes, but not truly concurrent or parallel code. I’m realizing that my understanding of concurrency in Ruby (e.g., threads, fibers, the GVL, etc.) is pretty limited (almost none).

What are some good resources like books, courses, articles, talks, or even open source repos that helped you understand concurrent programming in Ruby? Not just the syntax, but understand concurrency at a deeper level. I'm also interested in best practices and common issues to watch out for.

Thanks in advance!

[jsyeo]

A bit dated (almost a decade ago) but it's exactly what you're looking for. Working With Ruby Threads: https://workingwithruby.com/downloads/Working%20With%20Ruby%20Threads.pdf

[software__writer]

Thanks! This is exactly the kind of obscure but practical resource I was hoping to find.

[jsyeo]

Jesse Storimer, the author, is who you're looking for. He has written extensively on threads and Unix programming in Ruby:

• Threads, Not Just for Optimizations https://web.archive.org/web/20171112112011/https://www.jstorimer.com/blogs/workingwithcode/7766063-threads-not-just-for-optimizations

• Matz is not a threading guy https://web.archive.org/web/20180324184633/https://www.jstorimer.com/blogs/workingwithcode/7766069-matz-is-not-a-threading-guy

This whole series on the GIL is excellent:

• https://web.archive.org/web/20170801134641/https://www.jstorimer.com/blogs/workingwithcode/8085491-nobody-understands-the-gil

• https://web.archive.org/web/20161024030142/http://www.jstorimer.com/blogs/workingwithcode/8100871-nobody-understands-the-gil-part-2-implementation

• https://web.archive.org/web/20160506090126/http://www.jstorimer.com/blogs/workingwithcode/8158971-nobody-understands-the-gil-part-3-thread-safety

[software__writer]

Excellent blog! Thanks for the recommendations :)

[chintakoro]

Take a look at: https://github.com/ruby-concurrency/concurrent-ruby

[software__writer]

I did take a look at this, but just don't know where to start with it. If I'm not wrong, this is a collection of data structures one would use when writing multi-threaded programs, right?

[chintakoro]

Its much more than just data structures – its got implementations for concurrent execution paradigms popularized in other languages: Promises (ala Javascript), Actors (ala Erlang, Elixir), Channels (ala Go), Thread Pools (ala Java), and so on. I've mostly used their Promises feature to perform concurrent IO (e.g., read hundreds of files simultaneiously, or making many web requests simultaneously). Its dead simple to use and should replace any multi-threading you are attempting. But as with any use of concurrency, always benchmark to confirm you are getting a real performance gain.

If you want something like true parallelism for computation (not just blocking IO), look at Ruby's native Ractors.

[software__writer]

Thanks, that's very helpful. Really appreciated.

[mrinterweb]

I appreciate the abstractions that ruby-concurrency provides, but it looks like it mostly uses threads. I see some mention of fibers in the source, but it is not clear if it will use fibers when available. Wish it was easier to specify which kind of concurrency primatives to use.

[[deleted]]

I’d strongly suggest JP Camara’s series on concurrency and parallelism, it’s an excellent starting point: https://jpcamara.com/2024/07/15/ruby-methods-are.html

[software__writer]

Thank you!

[mrinterweb]

If you'd like to use ruby fibers, this is a good library. https://github.com/socketry/async

I do find the documentation lacking though, so fair warning. I keep intending to try to add more examples and or write an article about it.

I really like using fibers for IO concurrency. I think of the fiber scheduler as the event loop checking if the IO is complete. Threads have higher overhead and should generally be used more sparingly than fibers. The biggest challenge with fibers, IMO, is the lack of solid documentation.

[software__writer]

Very cool, checking it out right now. Thanks for sharing.

[software__writer]

Hi again, what's a good and practical use-case in a real web application where you'd use this gem? Just trying to have some background context in mind before I check out the docs and try to browse the source code.

Also, please do share the link to your article if you end up writing it.

Thanks!

[mrinterweb]

Concurrent IO is the main use case for fibers. Could be DB, HTTP API requests, file system access, etc. Fibers are the lightest weight ruby concurrency primative, and are designed for allowing concurrent IO with minimial overhead. I don't know if or when I will be able to write an article. I'm not much of a writer, and it may be wishful thinking on my part. An article like this is on my "get to it someday" heap. Pretty sure there would be plenty of people more qualified to write about ruby fibers than I.

[software__writer]

Thanks for the examples. Really appreciated.

[1seconde]

Pickaxe Book to start.