Weird borrowck issues with &mut &mut [u8]

Opened by Lily Ballard at 2024-04-25 04:12:40

I ran into some bizarre borrowck issues when trying to write a function that would modify a &mut &mut [u8]. The goal here was to make it simple to shove data into a [u8, ..512] by virtue of copying data into the corresponding &mut [u8] and then modifying the slice. I tried three different approaches, and they all had issues. The various errors are commented into the source.

(All 3 approaches require use std::vdc::MutableCloneableVector)

The first approach tried to define a function append(buf: &mut &mut [u8], v: &[u8]) that would copy the data into buf and then modify *buf to contain the new slice. This ran into an odd lifetime error where it thinks that I can't say *buf = buf.mut_slice_from(len). It claims the lifetime of buf is too short, but I don't see why that matters. I'm actually re-slicing *buf, and putting the result back into the same location that held the original slice, so I would expect it to have the same lifetime and, therefore, be valid.

fn one() {
    let mut line = [0u8, ..512];
    let mut buf = line.as_mut_slice();

    fn append(buf: &mut &mut [u8], v: &[u8]) {
        let len = buf.clone_from_slice(v);
        *buf = buf.slice_from_mut(len);
// error: lifetime of `buf` is too short to guarantee its contents can be safely reborrowed
//             ^~~
// note: `buf` would have to be valid for the anonymous lifetime #2 defined on the block at 7:45...
// note: ...but `buf` is only valid for the anonymous lifetime #1 defined on the block at 7:45
    }

    append(&mut buf, b"test");
    append(&mut buf, b"foo");
}

The second approach was to give both levels of indirection the same lifetime, e.g. append<'a>(&'a mut &'a mut [u8], v: &[u8]) to try and squelch the error. This didn't work because I wasn't allowed to reassign back to *buf, as it considered buf.mut_slice_from(len) to borrow it. I assume the borrow check is tied to the lifetime, which is shared at both levels, so borrowck thinks buf is borrowed when it's really *buf that's borrowed.

Curiously, it also decided I couldn't use &mut buf twice in the calling code, as it seemed to think it was already borrowed.

fn two() {
    let mut line = [0u8, ..512];
    let mut buf = line.as_mut_slice();

    fn append<'a>(buf: &'a mut &'a mut [u8], v: &[u8]) {
        let len = buf.copy_from(v);
        *buf = buf.mut_slice_from(len);
// error: cannot assign to `*buf` because it is borrowed
//      ^~~~
// note: borrow of `*buf` occurs here
//             ^~~
    }

    append(&mut buf, bytes!("test"));
    append(&mut buf, bytes!("foo"))
// error: cannot borrow `buf` as mutable more than once at a time
//         ^~~~~~~~
// note: previous borrow of `buf` as mutable occurs here
//         ^~~~~~~~
}

The third approach was to ditch &mut &mut [u8] entirely and try capturing buf in a closure instead. This gave some odd errors. First off, it kept referencing (*buf)[], and I don't know what it meant by that. Also, the first error here indicates that a borrow on a later line was blocking a borrow on an earlier line, which is quite bizarre. How can buf have been borrowed already when the later line is, well, later? It also considered the same reference to buf to consist of multiple mutable borrows.

fn three() {
    let mut line = [0u8, ..512];
    let mut buf = line.as_mut_slice();

    let append = |v: &[u8]| {
        let len = buf.copy_from(v);
// error: cannot borrow `(*buf)[]` as mutable more than once at a time
//                ^~~
        buf = buf.mut_slice_from(len);
// note: previous borrow of `(*buf)[]` as mutable occurs here
//            ^~~
// error: cannot borrow `(*buf)[]` as mutable more than once at a time
//            ^~~
// note: previous borrow of `(*buf)[]` as mutable occurs here
//            ^~~
// error: cannot assign to `buf` because it is borrowed
//      ^~~
// note: borrow of `buf` occurs here
//            ^~~
    };

    append(bytes!("test"));
    append(bytes!("foo"));
}

In the end, I couldn't figure out any way to accomplish what I wanted. It seems to me the first approach should have worked.

/cc @nikomatsakis

It is possible to do something like the following in safe Rust:

#![feature(collections)]
fn main() {
    let mut line = [0u8; 512];
    {
        let mut buf = &mut line[..];

        /// reslice is essentially equivalent to `*buf = &mut buf[len..];`.
        fn reslice(buf: &mut &mut [u8], len: usize) {
            let tmp = std::mem::replace(buf, &mut []);
            std::mem::replace(buf, &mut tmp[len..]);
        }
        fn append(buf: &mut &mut [u8], v: &[u8]) {
            let len = buf.clone_from_slice(v);
            reslice(buf, len);
        }

        append(&mut buf, b"test");
        append(&mut buf, b"foo");
    }
    assert_eq!(&line[..7], b"testfoo");
}

The error message you get if you try to write *buf = &mut buf[len..]; is still terrible:

<anon>:10:25: 10:35 error: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
<anon>:10             *buf = &mut buf[len..];
                                  ^~~~~~~~~~
<anon>:9:9: 11:10 help: consider using an explicit lifetime parameter as shown: fn reslice<'a>(buf: &'a mut &'a mut [u8], len: usize)
<anon>:9         fn reslice(buf: &mut &mut [u8], len: usize) {
<anon>:10             *buf = &mut buf[len..];
<anon>:11         }
error: aborting due to previous error

eefriedman at 2015-06-08 01:09:48

With a slight modification to @eefriedman 's example (clone_from_slice no longer returns its length), the example compiles, but panics with different length-d slices.

When trying to do the *buf version, you still do get that bad error message.
Steve Klabnik at 2017-01-03 17:57:46
This issue boils down to the following, I think.
```
fn append(buf: &mut &mut [u8]) {
    *buf = &mut buf[..];
}
```
Mark Rousskov at 2017-06-25 17:56:47
OK, so, it's only been like 3 years since the initial issue report. Sheesh. Let me explain a bit what is going on. I don't see an obvious fix at the moment.

First off, it's worth noting that while the original code does not compile, this variant (which uses a shared slice inside) does:
```
fn append(buf: &mut &[u8]) {
    *buf = &buf[..];
}
```
The reason for this is explained in my NLL-RFC draft, in particular when discussing Reborrow constraints and the notion of supporting prefixes.

When we do &mut (**buf)[..] (fully expanded), this is reborrowing the &mut [u8] slice found within another &mut reference. When you borrow a mutable reference from within another, you can only borrow for the lifetime of the outermost reference (which must be shorter than the others); otherwise, you can introduce unsoundness where the outermost loan ends, but your reborrow is still permitted, permitting access from multiple paths. In the original program, the signature is (expanded) &'a mut &'b mut [u8], where 'b: 'a, and hence we can only reborrow for 'a at most, but we would need to borrow for 'b.

If we adjust to <'a> &'a mut &'a mut [u8], we get a different error. That error arises because of the fact that &mut buf[..] introduces some temporaries. This error might be fixed by the rules proposed in the NLL RFC; I have to look at the desugaring, and I don't have time to go into that just now.
Niko Matsakis at 2017-07-17 16:06:03
Triage: NLL does not fix this.
Steve Klabnik at 2018-09-24 16:57:56
Issue incorrectly auto-closed. A commit in the subtree update of rust-analyzer "Fixes https://github.com/rust-lang/rust-analyzer/issues/11361" but GitHub misinterpreted it as this issue https://github.com/rust-lang/rust/issues/11361.
Alec Mocatta at 2024-01-22 14:03:56
Is this related?
Ricardo Fernández Serrata at 2024-04-25 04:12:40