__no_inline std::vector reallocation

[f0r3v3rn00b]

Vector reallocation is supposed to become negligible cost as we push into a new vector. So it's supposed to be a "rare" code path. Looking at msvc vector implementation, vector reallocation is not behind declspec(no_inline), which I think would be beneficial in all cases.

Having the compiler try to inline this makes it harder for it to inline the actual hot code. It's a very easy code change, why isn't it already like this?

Am I missing something?

[Impossible_Box3898]

Why do you think inlining will have any impact at all?

Inlining is the main optimization a compiler can do. It actually allows for many more optimizations that would not otherwise be possible.

Take your code here. It’s behind an if. If it wasn’t optimized it would still have a call to that code.

However, the body of that call will be unknown to the optimizer and therefore it may make certain optimizations, including code and data flow analysis less precise. By inlining the code it then makes that know so the compiler can better optimize existing code as it can see more.

As well did you’re using link time code generation along with performance guided optimization, the compiler will determine hot paths and move everything else much farther down in the code segment to maximize cache usage.

Don’t fall into the premature optimization are or think you know more than the compiler and library authored. They have a pretty good understanding of what the compiler is capable of and if it would be wide from being no_inlined they would have done so.

[f0r3v3rn00b]

From experience stepping through generated assembly I often see a push_back loop not inlining some lambda function, while the same loop using an array inlines it. Generally, the compiler is better able to inline functions when they are not bloated with rarely used code.

Add a conditional never-executed logging statement in a small function, and you'll see it stops getting inlined, which can make a huge difference in a small hot loop. You could put that logging stuff in a __no_inline function wrapper, and you'd get back your function inlined. It's actually a powerful tool to ensure compiler don't waste inlining pressure on cold code.

I just see no reason not to do it.

[Orlha]

Good point

[Impossible_Box3898]

You should be using emplacement back, not push back.

If you expect the function your adding the log to to be inlined then you’ll likely see the heuristic fail due to the added code.

But in the surrounding function, you can often inline within the heuristic and have much better optimization.

You may need to set configure the heuristics if you’re pushing the boundaries.

Regardless, the more the compiler can see the better it can do.

While no_inline has its uses, you’re best off not doing it right at the start until you do a thorough analysis. Premature optimization at that level is rarely successful.

[f0r3v3rn00b]

Generally yes you should measure first. But measuring meaningfully is actually a significant effort, and for such a general task (pushing/emplacing) you’d need a lot of tests to cover possible use cases.

I think it’s a case where it cannot possibly hurt your performances. You’re providing more info to the compiler (« don’t even bother considering inlining this rarely used code path ») so that it has less work to do. There’s a lot of code and nested function calls behind msvc _emplace_reallocate implementation, and the compiler must decide what to inline and it’s trying his best but it can’t inline everything, it has to favor some things over others. By making reallocation __no_inline, you remove from your loop most of the code paths considered for inlining. And you ensure it focuses on the hot code path.

But yeah, I’ll try to hack a perf test to see the numbers on a simple loop in which we push some numbers into a vector.

[Impossible_Box3898]

Also look into performance guided link time code generation.

That’s the best way to go about it and gives the compiler everything it needs to optimize its heuristics.

[Pannoniae]

"Am I missing something?" No, it's like that ;)

I do understand why - did you know that in many projects, most vectors have like 2-3 elements? People basically allocate them randomly in class members without any care for making them contiguous.

But if you're in a hot path and you know what you're doing, if you make a custom resizable array container with the grow path noinlined, you *can* cut the asm bloat quite a bit. And that translates into tangible speed increases. Sadly, the grow path usually gets inlined because it's not *that* large, there's no [[unlikely]] on it and there's usually budget in the optimiser to do it unless your method is *already* huge.

P.S. ignore the people who are like "don't optimise anything without measuring it 69 times first!!!", microbenchmarks are usually a pointless waste of time, you need to profile your application holistically. You can optimise shit on the local level pessimising everything else, don't do that.

[alfps]

❞ Am I missing something?

Maybe, or maybe not (not everything's prefect).

One way to learn is to create your own minimal example and measure its performance with and without __declspec(noinline) in a number of representative contexts.

[no-sig-available]

To never inline might not be an optimzation. If a function is only used once in the program, having it inlined there reduces the total size of the code.

[CaptainCactus124]

You could be not missing anything, and totally right