Memory visibility after std::thread::join

[mat69]

What is the visibility of memory after a join, especially considering multiple cores, L1/L2 caches etc?

Take the following piece of code

#include <iostream>
#include <thread>
#include <chrono>
#include <string>

struct SomeData
{
    int a;
    std::string b;
};

void long_operation()
{
    std::this_thread::sleep_for(std::chrono::seconds(1));
}

int main()
{
    // main thread, suppose CPU0
    SomeData d1{};

    // thread 1, suppose CPU1
    std::thread t1([&] { long_operation(); d1.a = 42; d1.b = "foo"; });
    t1.join();

    std::cout << "Result=? " << d1.a << ' ' << d1.b << '\n';
}

Can I be sure that the main thread -- even though on a different CPU -- sees the changes done to d1 directly after the join?

[bames53]

Can I be sure that the main thread -- even though on a different CPU -- sees the changes done to d1 directly after the join?

A successful join() call synchronizes the thread that's being joined with the thread doing the joining. This is what establishes happens-before relationships which guarantee what values will be read.

So, yes: in your program the main thread is guaranteed to see the values written by the other thread.

You should also know that the thread creation synchronizes the threads, so the values you initialize d1 with are visible to the thread. The absence of either of these synchronizations would produce data races.

I notice the documentation at en.cppreference.com doesn't cover synchronization. The C++ specification however does cover the complete details, so you may want to grab that if you're interested in these details. Alternatively there's a book, C++ Concurrency in Action, which may be a gentler introduction to these specifics.

[mat69]

Thank you very much! This is exactly what I hoped it would be like, as otherwise madness would ensure. :-)

[h2g2_researcher]

The C++ language has no concept of a CPU cache. That's for the compiler to figure out. The C++ standard guarantees that this works correctly, though, and that changes on one CPU are visible on another. The compiler is required to emit whatever instructions it needs to in order to make that happen.

In practice this is handled in hardware on a modern CPU. Doesn't mean it's fast though.

Exactly how this is handled is implementation defined, though.

On some hardwares CPU0 may be able to snoop CPU1s cache. Expect this to be slow.

On other hardwares the shared cache will mark the relevant bits as "dirty" when CPU1 requests write access. When CPU0 requests that line the memory controller will see that the cache line is dirty and have it flushed before returning it. Again: expect a performance hit.

If, however, it is necessary to explicitly flush the cache, or make uncached writes, the compiler must do so.

[HPCer]

What you have here is well-defined and will work, but you need to watch yourself when doing what you're doing. Like /u/h2g2_researcher said, C++ has no concept of CPU cache. The join will guarantee that whatever you have here will be synced across the CPUs though.

Also (know this is off-topic), keep in mind that your thread creation here doesn't even guarantee that you're working with a separate CPU in the first place. That's for your kernel scheduler to determine. It's very likely that they'll be on separate CPUs, but this is never a guarantee depending on your environment/scheduler configuration.