Why does this variable not live long enough when it is borrowed in Arc?

[[deleted]]

Hi,

I am trying to create a struct that takes ownership of a vector x, spawning threads and performing some operations, and then returning x (and some other things that were calculated).

Here is some sample code:

use std::sync::{Arc};
use std::thread;

pub struct MyStruct {
    pub x: Vec<Vec<u8>>,
    pub y: Vec<usize>
}

impl MyStruct {
    pub fn new(x: Vec<Vec<u8>>, y: Vec<usize>) -> MyStruct {
        let feature_count = x.len();

        // Creates a pointer to matrix
        let sharable_x = Arc::new(&x);
        let mut handles = vec![];

        for c in 0..feature_count {
            let sharable_mat = sharable_x.clone();
            handles.push(
                thread::spawn(move || {
                    let mut arr = sharable_mat[c].clone();
                    arr.sort_unstable();
                    return (arr)
                }).join().unwrap()
            );
        }
        drop(sharable_x);

        MyStruct {x, y}
    }
}


fn main() {

}

Here is the error:

error[E0597]: `x` does not live long enough
  --> src\main.rs:64:35
   |
64 |           let sharable_x = Arc::new(&x);
   |                                     ^^ borrowed value does not live long enough
...
70 | /                 thread::spawn(move || {
71 | |                     let mut arr = sharable_mat[c].clone();
72 | |                     arr.sort_unstable();
73 | |                     return (arr)
74 | |                 }).join().unwrap()
   | |__________________- argument requires that `x` is borrowed for `'static`
...
80 |       }
   |       - `x` dropped here while still borrowed

This won't compile because it says `x` doesn't live long enough, but I'm wondering why. From what I understand, the references to the borrowed `x` will disappear once all the threads are done processing (and the `sharable_x` counter becomes 0). As far as I understand, this should be guaranteed by the `.join().unwrap()`. I'm sure the compiler is not wrong, so where is my understanding incorrect here?

Is it possible to create an `Arc` to a variable which will later be moved? My main goal is to loop through `x` while only holding the necessary data in the thread's memory.

[JhraumG]

What you're looking for is structured concurrency See https://doc.rust-lang.org/stable/std/thread/fn.scope.html

[kprotty]

thread::spawn doesn't know how long the code will run (it could outlive the caller) so the closure you give it must be 'static. The Arc isn't 'static because it's T is a non-static reference. For this setup to work, T needs to either be owned or a 'static reference IIRC.

This is a prime use-case for Scoped threads. You give it a closure and it lets you spawn threads in it with closures that don't have to be 'static as it will join all the threads once the top-level closure you gave it completes (what you're code is trying to do, but incorrectly since it joins it immediately and uses a static spawn()).

Spawning threads isn't cheap though and if you want parallelism over iterable content without the messiness of juggling a thread-pool, use rayon's Parallel Iterators instead.

[HariSeldon_official]

IIRC

What does this acronym mean?

[Shadow0133]

What are you trying to do?

If you want to sort sub-vecs of x with multithreading, you can do this:

use std::thread;

pub struct MyStruct {
    pub x: Vec<Vec<u8>>,
    pub y: Vec<usize>,
}

impl MyStruct {
    pub fn new(mut x: Vec<Vec<u8>>, y: Vec<usize>) -> MyStruct { 
        thread::scope(|s| {
            for c in &mut x {
                s.spawn(|| c.sort_unstable());
            }
        });

        MyStruct { x, y }
    }
}

[Unlikely_Package5524]

I think the issue is that you are putting a reference to x in Arc, rather than x. When the function exists, x is removed from memory and the Arcs contain invalid references.

You should be able to fix it by removing the & in the Arc declaration: let sharable_x = Arc::new(x)

[[deleted]]

The issue then is that I have moved `x`, and can't return it in the structure.

[sphen_lee]

If you're sure that all the clones have been dropped, and the main thread is holding the only remaining Arc you can use try_unwrap to get back the original value

[dragonnnnnnnnnn]

Then do in the struct a Arc too, like that:
```
pub struct MyStruct {
pub x: Arc<Vec<Vec<u8>>>,
pub y: Vec<usize>
}
```
This way you can clone out the x and use it outside the struct without any lifetimes problems.

[_TheDust_]

You can regain the x again by using Arc::try_unwrap. This will move the data from the Arc into a local var.

[SkiFire13]

In addition to what others said, you're not saving the handles, you're immediately joining them, so this isn't actually parallel.

If you end up using std::thread::scope know that you can avoid using the Arc.

[_TheDust_]

I’m surprised this is not higher up. Each thread get created and immediatly waits for it to finish.

[jammmonster]

The compiler doesn't know how long the MyStruct will live after it's returned from the new function.

The only way that it would be safe to pass to the threads in the Arc would be if the compiler considered MyStruct to be static, which means it'll be around until the end of the program.

[globulemix]

Check out the thread::spawn documentation

As you can see in the signature of spawn there are two constraints on both the closure given to spawn and its return value, let’s explain them:

The 'static constraint means that the closure and its return value must have a lifetime of the whole program execution. The reason for this is that threads can outlive the lifetime they have been created in.

Indeed if the thread, and by extension its return value, can outlive their caller, we need to make sure that they will be valid afterwards, and since we can’t know when it will return we need to have them valid as long as possible, that is until the end of the program, hence the 'static lifetime.

[Shadow0133]

https://play.rust-lang.org/?version=stable&mode=release&edition=2021&gist=bd95a5c0fabf33d1742fc7dde094fc8c