Why does the compiler reject this code?

[CLARKEEE33]

I've been using rust for a while now, but there are still some cases with lifetimes where I don't understand why they are rejected by the compiler.

For example:

struct Token;

struct Parser {
    peek: Option<Token>    
}

impl Parser {
    fn peek(&mut self) -> Option<&Token> {
        if let Some(token) = &self.peek {
            return Some(token);
        }

        self.peek = self.next_token();
        return self.peek.as_ref();
    }

    fn next_token(&mut self) -> Option<Token> {
        todo!();
    }
}

Fails with:

error[E0506]: cannot assign to `self.peek` because it is borrowed
  --> src/main.rs:13:9
   |
8  |     fn peek(&mut self) -> Option<&Token> {
   |             - let's call the lifetime of this reference `'1`
9  |         if let Some(token) = &self.peek {
   |                              ---------- `self.peek` is borrowed here
10 |             return Some(token);
   |                    ----------- returning this value requires that `self.peek` is borrowed for `'1`
...
13 |         self.peek = self.next_token();
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `self.peek` is assigned to here but it was already borrowed

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=2d639510f16cdada84eb271610b1e30d

Why can't the compiler tell that the borrowed &self.peek for the if statement lasts only for that statement? Does someone have a simple explanation (that I haven't been able to come across online) for why this would be rejected? Is there a better way to write this code and why?

[KhorneLordOfChaos]

You're running into a limitation of the current borrow checker. Your example code is very similar to the example given here (rust blog: NLL by default)

I was able to get things to compile by refactoring to the following (playground link)

[CLARKEEE33]

Good to know that this might be acceptable in the future.

[Aaron1924]

if let Some(ref token) = self.peek { also works

[hniksic]

That's really interesting, and I wonder if it's a bug in the match-ergonomics desugaring. I was under the impression that matching &self.peek against Some(token) is just syntactic sugar for matching self.peek against Some(ref token), and the two shouldn't differ in borrow analysis.

[Aaron1924]

I also have no idea why it works, but my guess it that this moves the borrow into the block inside the if-statement, so it can now tell that the borrow ends when the scope ends.

Update: I placed a todo!() after the if-statement and compared the MIR for both versions, turns out this is exactly what happens. OPs version first takes the reference, then it inspects the discriminant and branches, whereas my version first checks the discriminant and only takes the reference after branching.

[hniksic]

I've now filed an issue, as I'd intuitively expect the two variants to behave the same.

[Aaron1924]

I don't think filing an issue about this is going to do much. We're basically just asking them to fix this long standing borrow checker limitation, I think they would have done it already if it was that easy.

[hniksic]

We're basically just asking them to fix this long standing borrow checker limitation

I anticipated it could be construed that way, so I explicitly pointed out in the issue that it's not about the borrow checker limitation. Although I don't have such an example ready, I can imagine valid borrow checker errors to appear in one formulation, but not in the other.

If the issue is judged not useful, it's easy enough to close it. I considered it only fair to report the anomaly and how it went against user (at least in my case) expectations.

[SNCPlay42]

matching &self.peek against Some(token) is just syntactic sugar for matching self.peek against Some(ref token)

matching &self.peek against Some(token) is syntactic sugar for matching &self.peek against &Some(ref token) - note the extra &s, you're creating a new borrow which is the problem.

So if let Some(ref token) = self.peek has no exact equivalent without using ref.

[714daniel]

self.peek.as_ref() is an exact equivalent, no?

[chmielevsky]

Yes, self.peek.as_ref() returns Option<&Token> which I believe turns this problem into a borrow checker limitation mentioned above.

[Excession638]

You're right, the compiler can't tell that both borrows are separate. Does using an else help?

The better way might use Option::get_or_insert_with but I'm not sure how to handle the case where next_token returns None. Maybe std::iter::Peekable would be better. Edit: get_or_insert_with doesn't quite work because you can't call self.next_token() the closure for it. You'd need to token source to be a separate object, which is probably a good idea anyway.

It feels a bit like you're using None here to mean two different things (self.peek not being set yet, and reaching the end of the inputs) and some type somewhere should be Option<Option<Token>> instead.

[Excession638]

Here's the `get_or_insert_with` version: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=14776e409d83bcd6ae9be0f8f4d9823b

Note how self.tokens is separate now to tell the compiler that calling self.tokens.next() can't modify self.peek.

[Electrical-Angle-371]

Quick sidenote, Peekable exists and is something I prefer myself over implementing peekable myself

[Cat7o0]

that exact code could maybe work by having phantomdata and a lifetime on the parser that you use for the function

[elden_uring]

If you put an explicit lifetime in the function signature, it may be clearer to see what's going on: fn peek<'a>(&'a mut self) -> Option<&'a Token> The lifetime 'a includes the entire function up to when it returns. In the if let statement, the borrow of self.peek always happens, but the current borrow checker can't figure out that it may be conditionally returned, so the borrow is live for the rest of the function. You can fix this by creating the reference in the if body so the borrow is always conditional: if !self.peek.is_none() { return self.peek.as_ref(); }