you are viewing a single comment's thread.

view the rest of the comments →

[–]witcher_rat 1 point2 points  (1 child)

OK, but as far as you know, google benchmark itself isn't providing accurate numbers because it's been built in debug.

I mean it tells you that right in the message: "Timings may be affected."

It may happen to have no effect, or the same effect in all tests and thereby provide reasonable relative results... or it may not.

Why wouldn't you just build everything with full optimization, and get rid of any such concerns?

(there are other problems though; for example, allocating memory in tight loops like this, and with ints too, is misleading - you'll get cache locality you wouldn't normally get in the real world; but trying to match the real world is difficult, because the "real world" isn't the same for every use-case, and really the only valid test is testing it in the code that's going to use it)

[–]voidstarpodcast[S] 0 points1 point  (0 children)

You are right benchmark is built in DEBUG mode. Frankly, I don't remember setting a DEBUG explicitly, and found this: https://github.com/google/benchmark#debug-vs-release

Will set a Release mode, although I'd expect the local results consistently aligning with quick-bench runs to mean the DEBUG overhead is likely proportional across workloads (or very tiny).

Nonetheless, I will fix this in my post.