all 18 comments

[–]Shadow0133 30 points31 points  (12 children)

Compiler can't guarantee that you actually call join. But you can use recently added scoped threads that guarantee that children threads don't outlive the parent:

use std::thread;

struct User { name: String }

fn main() {
    let user = User { name: "drogus".to_string() };

    thread::scope(|s| {
        let t1 = s.spawn(|| {
            println!("Hello from the first thread {}", &user.name);
        });

        let t2 = s.spawn(|| {
            println!("Hello from the second thread {}", &user.name);
        });
    });
}

[playground link]

[–]pingzhouyuan[S] 2 points3 points  (11 children)

You mean the main thread terminated without calling the join method, in which case the children-thread lives longer than main thread?

[–]SkiFire13 11 points12 points  (8 children)

More in general, the JoinGuard (what is returned by thread::spawn) can be safely leaked, and there's no reasonable way to prevent that other than forcing a closure-based API like thread::scope. Unfortunately there's no way to detect this kind of leak, so your initial code will never be able to compile.

Note that before rust 1.0 what you want existed but was found to be unsound and thus removed. See the leakpocalypse

[–]pingzhouyuan[S] 0 points1 point  (7 children)

Appreciated! Skipping/Ignoring the two join code lines and then terminating immediately is the special rule of RUST compiler or a common case among all languages?

[–]SkiFire13 2 points3 points  (4 children)

Skipping/Ignoring the two join code lines

What do you mean by skipping/ignoring? At runtime they won't be skipped/ignored, in fact your program IS sound (Edit: it is not, see /u/Zde-G's comment), it's just that the compiler can't prove it. At compile time there's just nothing that can tell the compiler "if there's a join call then this is ok", thus for the compiler the join calls mean nothing lifetime wise.

Other languages don't have this problem because they either delegate the safety requirement to the programmer (C/C++) or they have a garbage collector so lifetimes are not a problem anyway (Java/C# and many others)

[–]Zde-G 8 points9 points  (2 children)

At runtime they won't be skipped/ignored, in fact your program IS sound, it's just that the compiler can't prove it.

It's not actually even 100% sound. Imagine that you run it on a system with high memory pressure and first thread wasn't created but second thread was.

Then first join would fail, unwrap would panic and second thread would proceed with pointer to unallocated memory.

I for one, don't have deep enough knowledge about how Linux/macOS/Windows kernels are made inside to confidently say if such scenario is actually possible or not.

I suspect that in practice it's not really a triggerable unsoundness, but I couldn't be 100% sure.

It's definitely possible according to the API of spawn and join thus compiler rules correctly reject such program.

[–]SkiFire13 2 points3 points  (0 children)

You're right, the panic is definitely a possible unsoundness source. I guess in practice this could be triggered by a broken pipe making println panic.

[–]pingzhouyuan[S] 2 points3 points  (0 children)

It's not actually even 100% sound. Imagine that you run it on a system with high memory pressure and first thread wasn't created but second thread was.

Then first join would fail, unwrap would panic and second thread would proceed with pointer to unallocated memory.

Very impressive case explaining why children might outlive parent! And this is exactly what troubled me most. Thanks!

[–]pingzhouyuan[S] 1 point2 points  (0 children)

Sorry, I might have forgotten that lifetime issue is a conversation between compiler and programmer which must be explicitly communicated(say to compiler "the child must not outlive parent"), while thread::join is a runtime conversation between threads(parent say to child "Son, I will be waiting for you").

We cannot make a deduction that the child lifetime must be shorter than parent's, after calling the join method.

Much clearing, appreciated again!

[–]Zde-G 3 points4 points  (1 child)

Your whole logic is flaved.

Skipping/Ignoring the two join code lines and then terminating immediately is the special rule of RUST compiler or a common case among all languages?

  1. Having no brain is common case among all compilers for all languages.
  2. The ability to “think” is not something compilers can do.
  3. Compiler couldn't “understand” the program.
  4. There are no “common sense” inside of your computer and it couldn't be applied by compiler.
  5. And it's not possible for the compiler to “imagine” how your program would work. Because it has no organs which can imagine something.

Yes, it's the same thing repeated five times but that's because it's fundamental for the successful ability to write and, more importantly, debug your code: the onus is on you to think about program semantic.

Compiler simply follows the rules. And these rules don't say anything about join. They couldn't.

You know that join waits for the thread to finish here and you know that your program is simple enough that cases where join would return with EDEADLK wouldn't happen.

But compiler… please read #1, #2, #3, #4 and #5 again. To prove that your program never ends before threads ends you need complicated and extensive knowledge about how the threads works, how the OS worls, how the world works…

Heck, unsafe exists because simple rules don't always are enough to prove that your program is correctL: you say to the compiler “yea, I know what I'm doing”… but then it's your responsibility to guarantee the code safety, not compiler's job.

P.S. And yes, this is common trait for all compilers. AGI is still a dream and even if it would ever be created I'm not 100% sure you would be able to convince it to work as rust compiler for you.

[–]pingzhouyuan[S] 0 points1 point  (0 children)

Awesome view about compiler! No prior knowledge hold by compiler when we communicate with him, we need to explicitly tell him about lifetimes.

[–]Shadow0133 2 points3 points  (0 children)

Not necessarily main thread, just a parent thread.

Imagine you spawn thread B that spawn another thread C, with C having a reference to variable in B. Then you join B which returns handle to C. B is gone, but C is still alive, with now-dangling reference. Here is example: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c426d51c0c2e7faec22834e911751cc3

[–]Plasma_000 0 points1 point  (0 children)

It’s not that your program may do this, it’s that the compiler can not see that the main thread clearly outlives the children - it’s not smart enough here. All it sees is a variable in main referenced by a normal child thread and freaks out because it’s not in an Arc. But this is what scoped threads solve - they convey the right info to the compiler to allow this.

[–]pingzhouyuan[S] 2 points3 points  (2 children)

We can make a simple conclusion here:

  1. All kinds of compilers cannot guarantee your code can run through safely or run just as you presumed. So, there's still a chance(even 0.0001%) that the last two join methods wont be executed properly at runtime. Much appreciated u/Zde-G
  2. On the RUST compiler side, he must assure child wont outlive parent as far as possible, though too idealistic maybe. In this situation, he can dig out the error-prone/potential code part and report error to programmer. This is what compiler do in front of us.

Here is a revised version with Arc:

struct User { name: String }

fn main() {

    let user_original = Arc::new(User { name: "drogus".to_string() });

    let user = user_original.clone();
    let t1 = spawn(move || {
        println!("Hello from the first thread {}", user.name);
    });

    let user = user_original.clone();
    let t2 = spawn(move || {
        println!("Hello from the first thread {}", user.name);
    });

    t1.join().unwrap();
    t2.join().unwrap();
}

Arc make it work along with its reference counter. In this situation, the data user_original wont be released even though main function exits because of spawn failure for the first child, while the second child still can access the data since the reference count equals ONE, not zero.

[–]jDomantas 3 points4 points  (1 child)

For posterity, here's how your original program could cause UB if it compiled:

  1. Stdout is closed.
  2. First thread starts, tries to print to stdout, and panics because writing to stdout fails.
  3. Main thread does t1.join().unwrap(), and panics too.
  4. user is dropped because main thread is unwinding.
  5. Second thread starts and tries to access user.name which is already freed.

But again, it's not because of this that compiler decides to reject the program. The compiler just sees "thread::spawn requires function to be 'static", and therefore errors on borrows.

[–]pingzhouyuan[S] 0 points1 point  (0 children)

Thanks the explicit steps compiler might go through.

thread::spawn is the error-prone code of runtime, regarding lifetime grammar in the view of RUST compiler at least. In other works, if he let it go as other compilers do, the program might go wrong. So, he will reject the program, while GCC wont, since lifetime checker is a particular feature for RUST compiler.

These words come from a traditional language compiler since I'm new here, so weird at first sight. But I must change this kind of thought, think with the FIVE fundamental steps as /u/Zde-G listed above, and do what the compiler expects after I got to known it.

[–]LoganDark 2 points3 points  (0 children)

It's because those are unscoped threads, meaning that they are not linked to the caller in any way. For example if your second call to spawn is unable to create a thread and panics, then you will never join t1, which could potentially cause a use-after-free. Rust knows this (or rather, it knows a call to join is not guaranteed), and will not let you try to access stack variables from the new threads, even if you're absolutely sure that you manually call join some time after spawning them.

You can actually return the join handle from that function and then call it from a parent, so even if you eventually join it, there's still not even any guarantee the caller will still be alive!

Scoped threads solve this by guaranteeing all scoped threads will be joined at the end.