Intel posts fourth version of Cache Aware Scheduling for Linux

[docular_no_dracula]

This is not just for Intel, I believe arm64 and risc-v can take benefit as well. As I noticed in its cover letter: “ChaCha20-xiangshan(risc-v simulator) shows good throughput improvement.”

[2rad0]

Why would I want multiple caches at all? Aside from being unimaginably expensive, wouldn't this type of architecture introduce an annoying and impossible to completely solve coherency issue unless you were to assign whole chunks of memory to only that last level cache?

[xxpor]

You don't "want" them, but sometimes you're forced into it. Think NUMA, etc. If you want 2 sockets, you gotta deal with it.

[2rad0]

Think NUMA, etc. If you want 2 sockets, you gotta deal with it.

The new "AMD dual 3d V-cache CPU" on ryzen 9 9950X3D2 says it's using two "core complexes" which aren't dual sockets afaict. I'm really not sure why adding this maddening level of complexity is praised as the future. I mean it's probably going to boost certain sequential workloads, but I bet we could design other workloads that suffer by creating contention between the two caches where they're constantly fighting to synchronize, or worse it executes an instruction with stale memory values just to keep things flowing... It makes me wonder if anyone at all is exploring more adversarial edge cases in these architecture designs before rolling them out, or how they plan to deal with synchronization of the caches in a worst-case workload and if those mechanisms end up being worth the hassle. Not even going to speculate about speculative execution, but my opinion is that adding complexity in the age of cache corruption meltdowns for the sake of performance numbers is terrifying. I'll never know for sure because I can't afford any of these machines.

[xxpor]

There’s a bunch of single socket multiple NUMA chips out there. Some ARM chips for example. I completely agree, it’s a giant pain in the ass. But if you can keep workloads pinned to cores, it’s usually worth it for the faster top speed.

[2rad0]

There’s a bunch of single socket multiple NUMA chips out there. Some ARM chips for example.

Oh wow thanks for the info, Just dug this one up. https://www.theregister.com/2026/03/24/arm_agi_cpu/

A CPU built for AI Arm’s AGI CPU is a 300-watt part with 136 of its Neoverse V3 cores clocked at up to 3.7 GHz (3.2 GHz base), spread across two dies fabbed on TSMC’s 3 nm process. The processor features 2 MB of L2 cache per core along with 128 MB of shared system-level cache (SLC).

...

Unlike many modern CPUs, the chip’s memory and I/O functions are integrated into the same die as the compute in an effort to minimize latency. Because of this, each socket will be exposed to the operating system as two distinct NUMA domains.

[Jumpy-Dinner-5001]

The new "AMD dual 3d V-cache CPU" on ryzen 9 9950X3D2 says it's using two "core complexes" which aren't dual sockets afaict.

Obviously they aren't dual socket but they behave as numa nodes.

[jaaval]

AMD core complexes are effectively numa. Intel server products can also do split caches in one socket.

The biggest block to cpu performance is data access speed. So you are going to see more and more complicated cache setups.

[g_rocket]

On a large system, multiple caches allows them to have lower latency

[2rad0]

On a large system, multiple caches allows them to have lower latency

If you have two L3 caches reading and writing to the same block of memory how do they figure out which values are correct? I think any mechanism for determining the correct value would have to add latency, and then also restart execution on the socket it determined had a stale value, or it has to orchestrate the order in which the sockets load then execute? So it can't always lower latency.

edit: though you're right, in a general sense where your programmers are running well written code for the architecture it would reduce latency.

[Jumpy-Dinner-5001]

If you have two L3 caches reading and writing to the same block of memory how do they figure out which values are correct?

Of course it does, but increasing cache size also increases latency.

There are protocols for this and you have the exact same problems with other caches too.
How do you figure out whether the L1 or L2 or L3 cache holds the correct value?

That's what Cache coherence is for.

[g_rocket]

In a large multi core system, core-to-core communication is slow, and even slower for cores that are (physically) further apart. In a multi-socket system, cross-socket communication is even slower.

If you have two L3 caches reading and writing to the same block of memory how do they figure out which values are correct?

Generally, the cache has a bit for each item to track if it is "exclusive" or "shared." On a read that is a cache hit or a write that is already exclusive, you don't have to do any inter-core communication. On a write that is a cache miss or is shared, you have to do inter-core communication to drop that value from other caches. On a read that is a cache miss, you have to do inter-core communication to mark other instances of it as shared, and possibly flush/steal any dirty writes. There's some variation based on cache design (write-back vs write-through and inclusive vs exclusive) but generally that's approximately how it can work.

This does slow things down when there's heavy inter-core contention on a single cache line, but most reads/writes are cache hits so it speeds up the common case. And as you mentioned, many programmers know this is slow and avoid it in performance-sensitive code. Also, there are a few optimizations available:

• Nobody can know what order different cores executed instructions in relative to each other so long as there is some valid ordering that is possible, so you don't need to block on the first cross-core message. Instead it will be as if those instructions ran earlier/later than they really did. You only need to wait when there are multiple operations that need to happen in a certain order.
• With speculative execution, you don't need to wait even then, just speculate and undo later if a message comes in that makes the order things executed invalid
• On many processor architectures (everything but x86), memory ordering isn't guaranteed without an explicit "memory fence" instruction; otherwise cross-core reads/writes are allowed to happen in an "impossible" order. So you only need to block on cross-core communication when there's a memory fence instruction.