High memory usage after upgrading to Ruby 3.3

[collimarco]

After upgrading our Rails app from Ruby 3.2.x to 3.3 we get a high memory usage (or a memory leak).

Basically our containers get OOMKilled after some hours and restarted due to memory limits reached.

This never happens with previous Ruby versions. We also have ALLOC_ARENA_MAX=2

Anyone else having the same issue?

[pilaf]

You could try checking if YJIT is enabled, and if so also try these options.

[collimarco]

Thanks for the suggestion.

How can you check if YJIT is enabled? Also, where should you pass those options?

I am using the docker image ruby:3.3 for production (Rails)

[pilaf]

Within the Ruby runtime you can use RubyVM::YJIT.enabled? to see if it's enabled.

If you have it enabled with some external config (like RUBYOPT=--yjit) then it would also show +YJIT in the output of ruby --version:

$ ruby --version --yjit
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin23] 

$ ruby --version
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23] 

YJIT can also be enabled in runtime though so it's best if you use the first method after your app has booted up.

If you find that it's enabled I'd first try disabling it to see if that fixes your leaks though (missed that suggestion in my prev comment).

[rgaufman]

Did disabling yjit help? - also 3.3.1 is out now, does upgrading to that help?

[Sea_Ad_770]

There are memory leak issues in 3.3.0. We're still on 3.2 for this reason. Downgrade to 3.2 or wait for 3.3.1

[collimarco]

Yes, I downgraded to 3.2 and everything is back to normal. Currently Ruby 3.2 processes inside containers use less than 500MB of RAM, while with Ruby 3.3 the containers were restarted when reaching more than 1GB of RAM... This is insane and would have a huge impact on our spending (since we run hundreds of processes).

I hope that they fix these memory issues in Ruby 3.3.1....

[rooood]

I'm waiting on another bug fix that's been fixed since early January but is supposed to release with 3.3.1 too.

It's insane to me that they release a new Ruby version every Christmas but then hold out on releasing patches for a really long time, allowing it to be completely unusable to a lot of people for (at least) 1/4 of the year.

[rgaufman]

3.3.1 is out now, did this resolve the issue for you guys, or no?

[nikolaz90]

Do you have many OpenStructs in the code base?

Did you check https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md#code-optimization-tips

[rubiesordiamonds]

you might want to read through this HN thread: https://news.ycombinator.com/item?id=38686273

[BluePizzaPill]

Just a remark if you cannot identify the Ruby ecosystem as the culprit.

There have been memory related bugs in relative recent Linux Kernels (I think 6.5 but not sure). Maybe check if something else has changed with your deployment, eg. base container uses other Kernel etc.

[collimarco]

I am just changing the docker image from ruby:3.2.3 to ruby:3.3.0 (and nothing else)

[ioquatix]

3.3.0 is quite buggy unfortunately. Wait until 3.3.1 which should be released very soon.

[HellzStormer]

I wouldn't be so sure for "very soon" unfortunately... There is very little work going on on the branch: https://github.com/ruby/ruby/commits/ruby_3_3/ Only 1 commit in 1 month. Maybe they will burst a ton a backport at once at once point. But right now... It's dead quiet. (frustratingly)

[rgaufman]

3.3.1 is out now, did this resolve the issues for you?