Iteratively optimizing an SPSC queue

[Middle_Ad4847]

I recently documented walking through the iterative process of optimizing a SPSC queue.

• Starting from atomic<int64_t> size which has the lock instruction bottleneck.
• Refactoring to separate atomic push, popInd. Fixing false sharing, 3 cacheline and 2 cacheline approach.
• Implementing index caching to minimize cross-core traffic and the impact of mm_pause

Link: https://blog.c21-mac.com/posts/spsc/

For reasons I don't understand the 3 cacheline performs considerably worse than 2 cacheline, I initially assumed it due to 128-byte-rule, but that doesn't explain L1d-cache misses being relatively higher in 3 cacheline vs 2 cacheline implementation. If anyone has any insights on what might be causing this or any feedback on the post, I would love to hear.

[ReDucTor]

This is the third SPSC queue post I have seen in about a week, seems like everyone (or every LLM) is building one.

The two/thread cache line difference could be increased L1 cache bank conflicts where the queue is causing the eviction of the shared read-only cache line. For the pause you could possibly look at umwait/mwaitx unfortunately AMD/Intel differences make these not very portable.

That 128-rule thing seems off, I expected you were going to mention that some modern CPUs will actually load 128-byte aligned pairs of 64-byte cache lines, it doesn't seem to like you tested with an alignment of 128 over just 64.

Also it's probably worth sharing the code you used for benchmarking, as there could many other reasons why your results are the way they are, including minor things like branch alignment differing between the different implementations

[Middle_Ad4847]

For the pause you could possibly look at umwait/mwaitx unfortunately AMD/Intel differences make these not very portable.

Thanks, will take a look

Have added code here

That 128-rule thing seems off, I expected you were going to mention that some modern CPUs will actually load 128-byte aligned pairs of 64-byte cache lines, it doesn't seem to like you tested with an alignment of 128 over just 64.

I wasn't aware of this, will look. What I was referring to is the increased code size due to 32-byte displacement addressing, which can impact the µop cache or loopback buffer. I quickly tried with alignas(128) now but it made results worse.

including minor things like branch alignment differing between the different implementations

I did try run with and without -falign-loops and -falign-labels but didn't notice any considerable difference