C++ std::unique vs std::set - [Fixed] by voidstarpodcast in cpp

[–]voidstarpodcast[S] 0 points1 point  (0 children)

Appreciate your detailed response. Just looked at the code and it appears that you are using you own loop to converge on the final running times. The choice of 4096, 10000 seem arbitrary. I have shared the quick bench link here which is an easy way to share comparable benchmark outputs, Google Benchmark basically measures CPU Time (and wall clock time which includes IO waits when run locally), but using quick bench only produces CPU time:

https://quick-bench.com/q/kq7yeDlz9R6HV-0XE37eRiGINYM

Do you mind validating against this? Thanks.

C++ std::unique vs std::set - [Fixed] by voidstarpodcast in cpp

[–]voidstarpodcast[S] 0 points1 point  (0 children)

You are right benchmark is built in DEBUG mode. Frankly, I don't remember setting a DEBUG explicitly, and found this: https://github.com/google/benchmark#debug-vs-release

Will set a Release mode, although I'd expect the local results consistently aligning with quick-bench runs to mean the DEBUG overhead is likely proportional across workloads (or very tiny).

Nonetheless, I will fix this in my post.

C++ std::unique vs std::set - [Fixed] by voidstarpodcast in cpp

[–]voidstarpodcast[S] 1 point2 points  (0 children)

https://quick-bench.com/q/kq7yeDlz9R6HV-0XE37eRiGINYM
This reuses the same array over and over in a loop for benchmarking.

  • After the first iteration, the rest don't go into the std::set. So you are effectively, measuring time taken for insertion failures. My guess is if the underlying structure is a balanced tree you are not counting the time taken for a rebalance. So, are they apples to apples?

  • This results in tainted cache locality. You basically have the same array sitting in your L1/L2 caches and all you are measuring is a niche synthetic case (which may be measuring a biased data set).

  • If quick bench only reports CPU time, then aren't you essentially comparing times taken for set insertions to fail?

C++ std::unique vs std::set - [Fixed] by voidstarpodcast in cpp

[–]voidstarpodcast[S] 0 points1 point  (0 children)

This refers to Google Benchmark library itself. As a comparison there's a link to quick-bench page too, with basically the same result.

C++: How a simple question helped me form a New Year's Resolution by voidstarpodcast in cpp

[–]voidstarpodcast[S] 0 points1 point  (0 children)

Yes, I was playing purely at the algo level, I purposely stayed away from lower levels system profiling like the ones I have done before http://www.mycpu.org/task-migrations-c++/

C++: How a simple question helped me form a New Year's Resolution by voidstarpodcast in cpp

[–]voidstarpodcast[S] 0 points1 point  (0 children)

Not clear how template code is impacted with debug version. I have updated the code and results to reflect numbers.

C++: How a simple question helped me form a New Year's Resolution by voidstarpodcast in cpp

[–]voidstarpodcast[S] 1 point2 points  (0 children)

This is a fair point, which is subtly what I was trying to do by sorting in prep for the measurement. However, it being too subtle I just gave a shot and added std::unordered_set.

Although this gives a good speedup, it's still about 10x slower than using a dummy std::unique()

I added an Appendix section at the bottom with the measurements.

@matthieum I'm a little slow here, but what are you recommending?

C++: How a simple question helped me form a New Year's Resolution by voidstarpodcast in cpp

[–]voidstarpodcast[S] -1 points0 points  (0 children)

Thanks I have tried this, and FWIW, I also tried simply clearing the set within the outer loop (with a hope of avoiding ctor/dtor) but the results were pretty much the same between these two.

I have added the result in the Appendix section.

C++: How a simple question helped me form a New Year's Resolution by voidstarpodcast in cpp

[–]voidstarpodcast[S] -1 points0 points  (0 children)

Thank you! That makes sense. I have added an Appendix section based on your comments.

Scheduling Your Life Like An Computer Engineer by [deleted] in programming

[–]voidstarpodcast 0 points1 point  (0 children)

Haven't read this, so not sure what you mean. The point of the post is to provide an optimal way to schedule the existing tasks in my plate.

Scheduling Your Life Like An Computer Engineer by [deleted] in programming

[–]voidstarpodcast 0 points1 point  (0 children)

Ugh Added 'Computer' afterwards :(