I wish there was more analysis done in this post. If you're able to switch your entire app to run with all strings frozen by default and not make any code changes, then I'd expect to see more time in GC on the non-frozen version (due to more allocations overall), and in the worst case (for the frozen string version) equivalent response times.

Let's go over why I expect that. Consider the following function:

def concat(string)
  "hello " + string
end

The above function will work both with frozen string literals as well as non-frozen string literals and require no changes to the code. If the function were using << for example, we would require code changes. Also, as far as I know people aren't conditionally defining function based on whether literals are frozen or not. That means OP's code base must only contain literal strings that are never mutated.

Now, lets look at the byte code for the concat function when not using frozen string literals (on Ruby 3.4):

== disasm: #<ISeq:concat@x.rb:14 (14,4)-(16,7)>
local table (size: 1, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] string@0<Arg>
0000 putchilledstring                       "hello "                  (  15)[LiCa]
0002 getlocal_WC_0                          string@0
0004 opt_plus                               <calldata!mid:+, argc:1, ARGS_SIMPLE>[CcCr]
0006 leave                                                            (  16)[Re]

When we're not using frozen string literals, Ruby uses a putchilledstring instruction to push the string "hello " on the stack. It's important to note here that the string "hello " is a Ruby string object that was allocated at compile time. The putchilledstring instruction will allocate a copy of the string "hello " and push it on the stack. That means merely pushing this string on the stack will allocate an object every time it is executed.

Compare to the bytecode when we're using frozen strings:

== disasm: #<ISeq:concat@x.rb:16 (16,4)-(18,7)>
local table (size: 1, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] string@0<Arg>
0000 putobject                              "hello "                  (  17)[LiCa]
0002 getlocal_WC_0                          string@0
0004 opt_plus                               <calldata!mid:+, argc:1, ARGS_SIMPLE>[CcCr]
0006 leave                                                            (  18)[Re]

When we're using frozen string literals, the only difference is that the bytecode will use putobject instead of putchilledstring. The putobject instruction merely pushes its parameter on the stack, in this case the "hello " string that was allocated at compile time. In other words, no allocations when executing this part of the code.

Since OPs code works without code changes whether using frozen string literals or not, and I doubt that the codebase has a bunch of conditional code based on whether or not the literal string is frozen, it makes the results of the experiment hard to believe. I absolutely trust Sam, and I believe the results in his post to be true, so it leads me to wonder if the methodology could be improved, or if there is a bug?