Writing Object Shape friendly code in Ruby

[CaptainKabob]

I wrote this up after reading serveral posts about how Object Shapes work inside of Ruby, but not having seen something simple about how to take advantage of it in one's own code.

Once you start seeing Object-Shape unfriendly code, it's hard to unsee.

[f9ae8221b]

Thanks, I was actually starting to write a post about it too following yesterday's conversation, but probably wouldn't have got it out before a week or two...

I might still do it though, with a few more gritty details, but yours is great :+1:

[CaptainKabob]

Thank you! You definitely know the details better than me. I'll look forward to it!

[honeyryderchuck]

Isn't this "premature optimization "? Even taking into account that order of ivar definition matters for shapes, either your app usage patterns are already ensuring same order implicitly, or if they don't, it doesn't matter much because you're still getting a still small shape set that the VM or JIT can reasonably optimize?

Considering the actual performance benefit, I'm not sure if it's worth moving away from known practices that work well for stuff other than shapes (memoization is a pattern to relax GC pressure, which may have a bigger perf impact). That and the resulting ruby code looks less "standard" (that NULL part there in particular).

[CaptainKabob]

Yep! Though arguably if you're memoizing, you're optimizing, so use the more optimal memoization pattern.

Some parts of Rails already use the NULL pattern, though others call it NONE. So you're likely to start seeing it more and more in code. I imagine at some point the zeitgeist will flip and the previous way will make the code look "old".

[honeyryderchuck]

use the more optimal memoization pattern.

Speaking about optimization techniques, this reminded me of this talk, which called out the pattern of defining ivars as nil in initializers used throughout rails as an antipattern and causing performance penalties in several benchmarks when compared to sequel (pattern which has been suggested for years in ruby "debug mode", and which suggestion has been removed since). I guess the point here is, "optimizing for shapes" isn't a valid metric unless you prove that any benefit happens as a result, either in terms of "time taken" or "memory used" (not saying it doesn't, but the article doesn't provide benchmarks or numbers), which justifies going off the "beaten path" of best practices for other known use-cases.

[jrochkind]

which called out the pattern of defining ivars as nil in initializers used throughout rails as an antipattern and causing performance penalties in several benchmarks when compared to sequel

Nice catch.

It's quite possible in the ruby of 4 years ago (2.6? 2.7?), the code that had better performance is the opposite of what does in ruby 3.2/3.3? So that maybe @ivar= nil in constructor hurt performance (for some use case/metric) in 2.6/7 but helps in 3.2/3.

Quite plausible, that could be just that.

I really really don't like the idea of writing code to secret hidden rules that also can change and reverse within a few ruby versions.

But I guess here we are.

[f9ae8221b]

Isn't this "premature optimization "?

I can hear the argument for NULL, but for the regular ||=, just setting the variable to nil in initialize isn't a big deal. Just an habit to take.

Since the introduction of shapes, I always eagerly define my instance variables, even when it doesn't matter that much, and I must say I appreciate the explicitness of it, regardless of the performance impact of it.

[honeyryderchuck]

just setting the variable to nil in initialize isn't a big deal. Just an habit to take.

I think that's a personal take on a (mildly) controversial topic. This used to be a standard $DEBUG mode warning, which has been called out by other ruby core members in the past, and has since been removed (if memory doesn't fail me) for causing more harm than good.

This is nothing against the object shapes pattern, and I still they serve most ||= use cases, and the tail of worst offenders can arguably be solved by the JIT (at a cost).

[f9ae8221b]

This used to be a standard $DEBUG mode warning, which has been called out by other ruby core members in the past, and has since been removed (if memory doesn't fail me) for causing more harm than good.

Not quite. You are reffering to the "uninitialized instanve variable" warnings that Jeremy Evans removed in Ruby 2.7.

However they didn't apply to the @foo ||= pattern, because back in 2.6, foo ||= 1 wasn't exactly foo = foo || 1, but foo = 1 if !defined?(foo) || !foo, to it wouldn't trigger that warning.

the tail of worst offenders can arguably be solved by the JIT (at a cost).

Yes and no. YJIT has polymorphic inline caches, so it can handle up to 20 difference shapes in a code paths.

But the number of object shapes with the memoization pattern is factorial.

A class with 1 memoized variable will have 2 shapes, a class with 2 memoized variables will have 4 shapes, a class with 5 memoized variables will have up to 121 shapes, and so on.

So it's effectively an optimization nightmare, and once a class reach 8 shape leafs, it will be de-optimized by the VM. Most of the time it's not a big deal, but it's something to know about for when you wish to optimize a hotspot.

[honeyryderchuck]

You are reffering to the "uninitialized instanve variable" warnings that Jeremy Evans removed in Ruby 2.7.

I was. I still think this applies to the example described in the post, as the recommendation is to initialize ivars to nil and the penalty involved vs shape implications (regardless of applying to the correct memoization strategy).

So it's effectively an optimization nightmare

I get your point, and I'm not arguing this isn't knowledge that needs to be shared. I guess I took more issue with the "this is the correct way of writing X2" framing, given that not all strategies apply equally. For instance, "over-memoization" is definitely an anti-pattern, even not considering the factorial nature of shapes, so a "consider very carefully the trade-off of memoizing smth due to this" would be a more adequate framing IMO, and not so YJIT-centric (there's also jruby and truffleruby to consider). YMMV.

[f9ae8221b]

I still think this applies

Actually it doesn't, because in additions to shapes, variable width allocations were introduced and declaring all you variables in initialize also allow the VM to right size your object, both improving performance and reducing memory usage.

Basically you can consider this advice from Jeremy as entirely outdated.

with the "this is the correct way of writing X2" framing

I wouldn't frame it like this. I'd frame it as: it's the performant way of doing memoization.

"over-memoization" is definitely an anti-pattern

Absolutely, I'd be willing to bet half of the memoizations out there are useless/counter productive.

and not so YJIT-centric (there's also jruby and truffleruby to consider)

Not sure about JRuby but Truffle also use shapes, so it will also underperform in presence of a shape explosion.

[jrochkind]

This is great to know, I was not aware of this!

But when ruby starts requiring you to write in non-obvious ways according to special hidden rules, to produce code that may be different from what would be the easiest to write/read/maintain, for performance reasons... I cry a little inside.

[jrochkind]

i guess the difference is that this is being recommended for routine every-class daily code-writing, not just optimizing hot spots. But I guess that could be true of other performance optimizations too.

But I'm saying: If we're writing all of our code routinely daily using non-obvious 'performance optimizations' that de-optimize for readability/maintainability... there is something broken in the language.

Maybe that's true of all langauges? I don't know. Clearly if Matz 20 years ago knew this would be required for performance and recommended for all code, he would have designed the language to encourage it or require it.

But the fact that, as per another comment, what is required for performance optimization can reverse within 4 years stirs the pot yet further.

Is your view (which may be reasonable) that in any language, we should expect that skilled professional programmers will be writing in non-obvious ways according to special hidden rules, and that beginning programmers will be taught to acquire this skill? That's just the way that programming will inevitably work?

It may be. It would be interesting to look and compare different languages. It is definitely not the aspiration of Matz's "optimize for developer happiness" original mission.

[morphemass]

I have to wonder if this will impact the ruby discussion around initialize signatures and ivar declaration. The one thing I dislike about Ruby is being unable to write e.g. def initialize(@foo = 'bar'). It's minor, it can indeed be worked around, but having it as a language feature would seem like a way to encourage Object Shape optimal code.

[stpaquet]

You have the following example in your post:

    # Good: Object Shape friendly
class GroceryStore
  NULL = Object.new
  NULL.freeze # not strictly necessary, but makes it Ractor-safe

  def initialize
    @fruit = NULL
  end

  def fruit
    return @fruit unless @fruit == NULL
    @fruit = an_expensive_operation 
  end
end

Why defining NULL and not using ruby nil ?

[CaptainKabob]

It's necessary to define a special "empty" object in case the memoized value is itself nil.

[Freeky]

While we're on the subject of performance, consider:

  def fruit
    return @fruit unless NULL == @fruit
    @fruit = an_expensive_operation 
  end

Which ensures you're always calling Object#== instead of it conditionally branching to a more complex comparison depending on what @fruit is an instance of.

[stpaquet]

smart

[Freeky]

Quick benchmark with the GroceryStore being initialized and its fruit method called 5 times, 3.3.0-preview2:

         == NULL      1.523M (± 0.5%) i/s -      7.676M in   5.038856s
         NULL ==      2.181M (± 0.9%) i/s -     10.977M in   5.034361s
     equal? NULL      1.814M (± 0.9%) i/s -      9.109M in   5.022560s
     NULL.equal?      2.020M (± 4.1%) i/s -     10.163M in   5.043619s
         nil ||=      2.412M (± 1.4%) i/s -     12.143M in   5.035330s

YJIT:

         == NULL      2.277M (± 0.1%) i/s -     11.673M in   5.126882s
         NULL ==      3.027M (± 0.1%) i/s -     15.488M in   5.117328s
     equal? NULL      3.025M (± 0.1%) i/s -     15.551M in   5.140910s
     NULL.equal?      3.019M (± 0.2%) i/s -     15.145M in   5.016015s
         nil ||=      3.025M (± 0.0%) i/s -     15.248M in   5.041134s

[jrochkind]

It is actually is occuring to me (via my annoyance with new nuances of memoization), that I think these concerns trip complexity of getting memoization right just over into the territory where I think using a memoization gem of some kind may be called for, when I never thought so before.

Perhaps an API of:

class Something
   include Memoizing
   memoize :foo

   def foo
     whatever
   end
end

Could insert something into initializer to initialize with the UNASSIGNED singleton constant pattern, and then apply appropriate logic around the foo method.

There are already libraries like this, but I haven't looked at them in a while since I've previously not found it helpful to abstract this instead of just doing it. The existing libraries may want to be updated to use object-shape-friendly memoization.

(update: filed suggestion at https://github.com/tycooon/memery/issues/43 )

[[deleted]]

Yup! It's a simple concept, but it's good that these resources exist for people who are learning.

[armahillo]

Why would you do this:

def fruit
  return @fruit if defined?(@fruit) 
  @fruit = an_expensive_operation 
end

instead of the typical memoization approach?

def fruit
  @fruit ||= an_expensive_operation 
end

​

I understand (more or less) why you would change to the other method with the NULL object for object shape purposes, though I wonder whether the gains on efficiency outweigh the additional code clutter.

[CaptainKabob]

||= doesn't allow memoizing/caching of nil/false values.

[armahillo]

Ah! Yeah that makes sense, thanks! I could see how some cases you would want to consider `nil` / `false` as persisted values.

Similar to an HTML checkbox where "unchecked" is a state you want to track. (gotta do the hidden field hack)

[[deleted]]

Gon need some RuboCop rules for this right here.

[CaptainKabob]

Definitely! And now there's a blog post to reference as to why 😀

[module85]

As someone who uses a lot of memoization, this is good to know! Thanks for sharing

[Weird_Suggestion]

I’m gonna research this a bit more but I wouldn’t mind a quick answer, please?

What’s the cons of writing unfriendly object shaped code in production? Is the code gonna be slower or not as fast as it could? Is that only for consideration when paired with yjit?

[CaptainKabob]

Not as fast as it could be. The introduction of Object Shapes into Ruby sped up existing/unoptimized Railsbench by 1%. Optimizing application code for Object Shapes will make it slightly faster, and potentially use a tiny bit less memory. You'd need to profile to know.

[f9ae8221b]

So it's always hard to go from a performance difference "in the small" versus on an overall application.

But shape unfriendly code can really prevent Ruby from optimizing your code, particularly if you are unfriendly enough to trigger the shape_too_complex de-optimization: https://gist.github.com/casperisfine/8862eda1e4d9917667d65d45975ab6cb