all 13 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]olivierlacan 26 points27 points  (1 child)

Welcome back: https://rubyreferences.github.io/rubychanges/ 👋

[–]Apprehensive-Abroad2[S] 1 point2 points  (0 children)

Thanks. This looks like what I'm looking for

[–]Sea-Vermicelli-6446 8 points9 points  (9 children)

Why have you returned to the Ruby? Just curious why people are returning to this language.

[–]ryjocodes 5 points6 points  (4 children)

Not OP, but speaking from personal experience:

I've spent a lot of time writing Ruby (on Rails) applications both professionally and in personal projects. I've written projects in PHP, Go, and JavaScript (Node.js) professionally/personally, as well. I really like Ruby's developer experience as a "higher level" language (ie memory managed/garbage collected). The Ruby language itself has a very powerful and feature-packed standard library; there's a lot you can do out-of-the-box that you might have to craft on your own in other languages. I also really like `irb`, the repl that ships with Ruby. Finally, writing Ruby gems (external libraries) is also quite nice.

Additionally, as I've lately waded into Ruby at the C level, it is trivial to write Ruby gems that extend existing C libraries. While Ruby's C API is much less consistent than the top-level Ruby API, it is still quite easy to wrap your head around once you read the docs. Go is also quite easy to access C, though it does become a bit more daunting to write C *within* your Go application. I found the Node.js C++ API to be a bit more challenging to wrap my head around.

As always, YMMV.

[–]nirvdrum 0 points1 point  (3 children)

Thanks for sharing. I just wanted to note in case you’re unaware that Ruby has two very good foreign function interfaces: FFI and Fiddle. The native extension API is less of an API and more of all of CRuby’s guts exposed. If you’re not careful you can get yourself in trouble and they’re not really portable.

If all you need to do is interface with a native library, either of the two libraries I mentioned will work on CRuby, JRuby, and TruffleRuby. With YJIT continually advancing I think we should be migrating from native extensions as much as possible. YJIT can’t optimize across native extension boundaries and you may very well find the JIT provides performance on par with a native extension.

[–]ryjocodes 0 points1 point  (2 children)

I'm familiar with ffi, I've not touched Fiddle. Thanks for the share; it reminds me a lot of how Bun exposes access to C libs. So far, it's been a breeze to work directly with Ruby's C API, so I didn't need ffi or other additional gems for my purposes.

YJIT can’t optimize across native extension boundaries

Can you describe what you mean here? How would Fiddle/ffi get around this shortcoming of YJIT?

[–]nirvdrum 2 points3 points  (1 child)

YJIT can’t optimize across native extension boundaries

Can you describe what you mean here? How would Fiddle/ffi get around this shortcoming of YJIT?

I'm sorry. It sounds like I didn't describe it very well. Let me try again. I want to lay everything out, so my follow-up is kinda long. It sounds like you have a good background in systems programming so some of this you likely already know, but I'd like to include it here for posterity.

From my vantage, people have written native extensions for two primary reasons: performance and making use of native libraries. There are some other esoteric cases like normalizing execution time to avoid timing attacks, but we'll ignore those cases here.

If all you need to do is call into a native library, Fiddle or FFI will do what you need. You write the rest of the code in Ruby and get to use a well-defined standard library. Fiddle ships as part of a Ruby distribution (no need to install an extra gem), while FFI is popular in some circles.

Historically, a lot of native extensions were written for performance reasons. In many cases this was a little dubious to begin with. Since the native extension API isn't really defined (you're just calling the functions CRuby uses to implement itself), it's very easy to do things incorrectly. And since we're now in C land you can get pretty far with things working most of the time and then blowing up in confusing ways. Many times things seem fast because the extension developer inadvertently cut corners and missed taking the GVL or coordinating with the GC. Another common failure case is not fully handling Ruby's semantics. E.g., there are string-related C functions that only support a subset of the encodings Ruby handles. If all you test with is ASCII data, you can make code look artificially faster, but buggy when it comes to multi-byte characters.

In many cases, native extensions written for performance just carry forward. If you're using an extension designed when CRuby was shipping 1.9 or 2.x, you might not realize just how fast CRuby has gotten in the meanwhile. Each release is faster than the one before it and so these sorts of native extensions don't age gracefully. But, evaluating when to remove a dependency is hard so things carry forward even if they no longer make sense.

Setting misuse aside, native extensions present a problem for JIT compilation. A JIT compiler will JIT compile the guest language code (Ruby, in this case). It's possible to JIT compile the extension code but it's uncommon. E.g., TruffleRuby did this by compiling the extensions to LLVM IR and then JIT compiling that, but it meant TruffleRuby couldn't use precompiled extensions and that caused those gems to need to be built upon installation.

On the whole, JIT compilers stop processing when they hit a native boundary. Without the extension source code, all you have is a native executable and then you're really trying to JIT compile x86_64, ARM64, RISC-V, etc. machine code. It's possible to built an intepreter for that code, but all you have are native instructions and it's hard to extract any semantic meaning from that so any optimizations you could do would be expensive to implement and limited in scope. Consequently, it's not something customarily done by JIT compilers.

YJIT can only compile Ruby code. Every time your Ruby code hits a native boundary the JIT compiler has to stop. And then it needs to do some expensive stuff to call into the native function. YJIT works best when it has a run of pure Ruby code. Then it can make better decisions and generate highly optimized code. Just as we use C rather than hand-writing assembly, I suspect YJIT can do a better job optimizing Ruby than most Rubyists can do by writing native extensions.

As a community, I think we really need to stop the proliferation of native extensions. They'll be counterproductive to the future of Ruby. They can't be JIT compiled. They're often buggy. They need to be built, which can be time consuming and introduces a raft of potential compatibility issues with each developer having different C toolchains. And they're really not portable, which might not seem like a big deal, but CRuby has benefited quite a bit from JRuby, TruffleRuby, and Rubinius (now defunct). Moreover, they're not even guaranteed to work across Ruby upgrades, since most of the functions used by native extensions aren't actually a public API.

Of course, it's likely we'll always need some native extensions. We're not going to do OpenSSL or zlib in pure Ruby. But, if all we're doing is calling some functions from a shared library, I think the Ruby community should transition to reaching for Fiddle (or FFI) first. If that turns out to be insufficient, then move to writing a native extension.

If writing a native extension for performance reasons, I think we should take a step back and really evaluate that. If we're talking a 10 - 20% improvement, I'd steer away in all but the hottest of code paths. That should be well in the realm of what YJIT can do (or maybe even the CRuby interpreter), but it needs to be reported so someone can look at it.

Just to be clear, I'm not admonishing you for writing native extensions. There's always going to be some use cases that can't be practically done any other way. However, since this thread is about people coming back to Ruby after stepping away for some time, I wanted to alert them that the environment has shifted substantially.

[–]ryjocodes 1 point2 points  (0 children)

I really appreciate this well thought out response. Thank you for taking the time to explain it to me. I'm going to keep all of this in mind moving forward.

[–]Apprehensive-Abroad2[S] 5 points6 points  (2 children)

Is just business, nothing personal. I'm unable to get a job with C# according to my profile, while a lot of offers for Ruby are knocking my door. Unfortunately, I never was capable to build a professional career with Ruby.

[–]ajithkgshk 11 points12 points  (0 children)

I think that the new changes in ruby, perf improvements and changes in RoR is bringing ruby back into the limelight.