impl-trait return type is bounded by all input type parameters, even when unnecessary

Opened by Arnav Singh at 2024-05-20 20:55:29

rustc 1.20.0-nightly (f590a44ce 2017-06-27)
binary: rustc
commit-hash: f590a44ce61888c78b9044817d8b798db5cd2ffd
commit-date: 2017-06-27
host: x86_64-pc-windows-msvc
release: 1.20.0-nightly
LLVM version: 4.0

trait Future {}

impl<F> Future for Box<F> where F: Future + ?Sized {}

struct SomeFuture<'a>(&'a Client);
impl<'a> Future for SomeFuture<'a> {}

struct Client;
impl Client {
    fn post<'a, B>(&'a self, _body: &B) -> impl Future + 'a /* (1) */ {
        SomeFuture(self)
    }
}

fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
    client.post(&[username])
}

fn main() {
    let client = Client;
    let _f = {
        let username = "foo".to_string();
        login(&client, &username)
    };
}

Since SomeFuture borrows 'a Client, I'd expect impl Future + 'a to be the correct return type, but it gives this error:

error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
  --> src\main.rs:21:16
   |
21 |    client.post(&[username])
   |                  ^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the function body at 20:1...
  --> src\main.rs:20:1
   |
20 | / fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
21 | |  client.post(&[username])
22 | | }
   | |_^
note: ...so that expression is assignable (expected &str, found &str)
  --> src\main.rs:21:16
   |
21 |    client.post(&[username])
   |                  ^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 20:1...
  --> src\main.rs:20:1
   |
20 | / fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
21 | |  client.post(&[username])
22 | | }
   | |_^
note: ...so that the type `impl Future` will meet its required lifetime bounds
  --> src\main.rs:20:53
   |
20 | fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
   |                                                     ^^^^^^^^^^^^^^^^

error: aborting due to previous error(s)

Changing _body to have an explicit lifetime like _body: &'b B where 'b is independent of 'a or where 'a: 'b does not change the error. This and the original error make it seem that returning an impl trait is somehow causing the _body parameter to get the 'a lifetime, even though it's clearly unused, let alone used in a way that it would require the 'a lifetime.
Changing (1) from impl Future + 'a to SomeFuture<'a> fixes it.
Changing (1) from impl Future + 'a to Box<Future + 'a> and returning Box::new(SomeFuture(self)) fixes it.

rustc 1.20.0-nightly (f590a44ce 2017-06-27)
binary: rustc
commit-hash: f590a44ce61888c78b9044817d8b798db5cd2ffd
commit-date: 2017-06-27
host: x86_64-pc-windows-msvc
release: 1.20.0-nightly
LLVM version: 4.0

trait Future {}

impl<F> Future for Box<F> where F: Future + ?Sized {}

struct SomeFuture<'a>(&'a Client);
impl<'a> Future for SomeFuture<'a> {}

struct Client;
impl Client {
    fn post<'a, B>(&'a self, _body: &B) -> impl Future + 'a /* (1) */ {
        SomeFuture(self)
    }
}

fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
    client.post(&[username])
}

fn main() {
    let client = Client;
    let _f = {
        let username = "foo".to_string();
        login(&client, &username)
    };
}

Since SomeFuture borrows 'a Client, I'd expect impl Future + 'a to be the correct return type, but it gives this error:

error[E0700]: hidden type for `impl Future + 'a` captures lifetime that does not appear in bounds
  --> src/main.rs:16:5
   |
15 | fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
   |                                            ----     ---------------- opaque type defined here
   |                                            |
   |                                            hidden type `impl Future + 'a` captures the anonymous lifetime defined here
16 |     client.post(&[username])
   |     ^^^^^^^^^^^^^^^^^^^^^^^^
   |
help: add a `use<...>` bound to explicitly capture `'_`
   |
15 | fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a + use<'a, '_> {
   |                                                                      +++++++++++++

Changing _body to have an explicit lifetime like _body: &'b B where 'b is independent of 'a or where 'a: 'b does not change the error. This and the original error make it seem that returning an impl trait is somehow causing the _body parameter to get the 'a lifetime, even though it's clearly unused, let alone used in a way that it would require the 'a lifetime.
Changing (1) from impl Future + 'a to SomeFuture<'a> fixes it.
Changing (1) from impl Future + 'a to Box<Future + 'a> and returning Box::new(SomeFuture(self)) fixes it.

lcnr at 2024-12-20 12:26:26

From some experimentation, it seems to be because fn foo<T>(_: T) -> impl Bar compiles to something like fn foo<T>(_: T) -> impl Bar + 't, where 't is the lifetime of T. (I don't think this relationship can be expressed in regular Rust code, though Ralith on IRC suggested one could consider the anonymous type to contain PhantomData<T>)

Thus in the original code fn post<'a, B>(&'a self, _body: &B) -> impl Future + 'a effectively forces the return type to be bounded by B's lifetime in addition to 'a. This is why changing _body: &B to _body: B does not change anything, nor does annotating the parameter as _body: &'b B for an unconstrained 'b

In light of that, here's a smaller repro:
```
#![feature(conservative_impl_trait)]

trait Tr { }

struct S;
impl Tr for S { }

fn foo<T>(_t: T) -> impl Tr {
    S
}

struct S2;

fn main() {
    let _bar = {
        let s2 = S2;
        foo(&s2)
    };
}
```
which complains that s2 is dropped while still borrowed because the compiler thinks it needs to live as long as _bar.
Arnav Singh at 2017-10-10 07:50:15
This is an intentional restriction. See RFC 1951 for the reasoning.
Taylor Cramer at 2018-01-05 20:51:25
Sure. So can there be a way that I can convince the compiler that Client::post's and foo's results don't depend on all the inputs? Maybe it can be made to not override explicit lifetimes like it does in the Client::post example? (I recognize this won't help for the foo example since there is no way to ascribe a lifetime to T.)

The reasoning in the RFC is that eventually there will be syntax that provides control over which lifetime are and are not part of the returned existential ("Assumption 1"). But for the OP example where there already is an explicit lifetime and bound so it could be made to work today.
Arnav Singh at 2018-01-05 21:50:36
Oh, I understand what you're saying now. Yes, if the return type contains an explicit lifetime bound, the compiler should be able to understand that the returned type outlives that lifetime bound. Currently, it cannot do that if there are type parameters involved. This should be fixed. Thanks for the report!
Taylor Cramer at 2018-01-05 22:11:27
This is fixed by existential_type
- existential type PostFuture<'a>: Future + 'a; fn post<'a, B>(&'a self, _body: &B) -> PostFuture<'a> {
- existential type SomeTr: Tr; fn foo<T>(_t: T) -> SomeTr {
The fact that existential types intentionally don't have the "generic lifetime inheriting" behavior that impl trait has is documented here.

Updated 2023-10-09 for current syntax:
- type PostFuture<'a> = impl std::future::Future<Output = ()> + 'a; fn post<'a, B>(&'a self, _body: &B) -> PostFuture<'a> { Note that the let f = client.post(); async { f.await } construction is required. Just client.post() or let f = client.post(); f fail with "opaque type's hidden type cannot be another opaque type from the same scope"
- type SomeTr = impl Tr; fn foo<T>(_t: T) -> SomeTr {
Arnav Singh at 2018-08-06 16:13:03
@cramertj @pnkfelix This should probably be closed for the same reason as https://github.com/rust-lang/rust/issues/53450, if I'm not mistaken?
Alexander Regueiro at 2018-11-19 00:13:19
No, this is still a bug. The fact that existential type provides a workaround does not mean that the code here shouldn't work-- if you say that your return type outlives some lifetime, then it shouldn't also be bound by the lifetime of a type that cannot be contained in the return type.

e.g. this doesn't compile today and shouldn't:
```
trait X {}
impl<T> X for T {}
fn foo<'a, T>(x: &'a u8, t: T) -> impl X + 'a {
    (x, t)
}
```
You have to explicitly add T: 'a in order for the return type to satisfy the impl X + 'a bound. If T didn't already outlive 'a, it couldn't appear in the return type.
Taylor Cramer at 2018-11-19 22:45:08
@cramertj Okay, so this is an issue with a lifetime inference, right? (Not actual bounds checking.) Would you mind writing up mentoring instructions so someone can tackle this? (Maybe even me.)
Alexander Regueiro at 2018-11-19 23:14:43
@nikomatsakis Can we do something about this soon you think? :-) Seems kind of urgent to me, though perhaps this affects me more than most users.
Alexander Regueiro at 2018-11-24 04:10:58

Just to check, this is the same issue right?

use std::path::Path;

trait RR {}
impl RR for () {}

fn foo<'a>(path: &'a Path) -> impl RR + 'static {
    bar(path)
}
fn bar<P: AsRef<Path>>(path: P) -> impl RR + 'static {
    ()
}

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0a65678ab18f34888291c9c514ec2834

gives

error: cannot infer an appropriate lifetime
 --> src/lib.rs:7:9
  |
6 | fn foo<'a>(path: &'a Path) -> impl RR + 'static {
  |                               ----------------- this return type evaluates to the `'static` lifetime...
7 |     bar(path)
  |         ^^^^ ...but this borrow...
  |
note: ...can't outlive the lifetime 'a as defined on the function body at 6:8
 --> src/lib.rs:6:8
  |
6 | fn foo<'a>(path: &'a Path) -> impl RR + 'static {
  |        ^^
help: you can add a constraint to the return type to make it last less than `'static` and match the lifetime 'a as defined on the function body at 6:8
  |
6 | fn foo<'a>(path: &'a Path) -> impl RR + 'static + 'a {
  |                               ^^^^^^^^^^^^^^^^^^^^^^

(I also think the suggestion is misleading - won't adding a 'a after 'static' be a no-op since it will choose the longer of the two?)

Aidan Hobson Sayers at 2018-12-23 21:22:37

@aidanhs Yes to both (well, the answer to the second question is a bit more complicated, but "it won't work" is correct ;) ).
Taylor Cramer at 2018-12-26 18:02:46

Also encountered this issue with the following (minimised) code. Also found a workaround by specifying a dummy trait and moving the (static) lifetime there.

Errors

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=56e0e8506c1b32ee44d471d5cb6ed215

trait Parser2 {
    type Input;
    type PartialState;
}

struct Test<I>(::std::marker::PhantomData<fn(I)>);

impl<I> Parser2 for Test<I> {
    type Input = I;
    type PartialState = ();
}

fn line<'a, I>() -> impl Parser2<Input = I, PartialState = impl Send + 'static> {
    Test(::std::marker::PhantomData)
}

fn status<'a, I>() -> impl Parser2<Input = I, PartialState = impl Send + 'static> {
    line()
}

fn main() {
}

Works

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=74f4abad271f12df1d79a133007ee07f

trait Parser2 {
    type Input;
    type PartialState;
}

struct Test<I>(::std::marker::PhantomData<fn(I)>);

impl<I> Parser2 for Test<I> {
    type Input = I;
    type PartialState = ();
}

trait Static: Send + 'static {}
impl<T> Static for T where T: Send + 'static {}

fn line<'a, I>() -> impl Parser2<Input = I, PartialState = impl Static> {
    Test(::std::marker::PhantomData)
}

fn status<'a, I>() -> impl Parser2<Input = I, PartialState = impl Static> {
    line()
}

Markus Westerlind at 2019-01-09 16:18:10

@nikomatsakis Would be curious to get your thoughts on this along with the other not-too-dissimilar covariant lifetimes issue.
Alexander Regueiro at 2019-01-09 18:27:15
I've often found it pretty useful to model impl Trait in terms of associated types. If we include the effects of the default keyword, in fact, we get a very close modeling (without the inference), except for "auto trait propagation". I think this modeling is relevant here.

the model

Basically an impl Trait instance like
```
fn post<'a, B>(&'a self, _body: &B) -> impl Future + 'a { .. }
```
can be modeled as if there were a "one-off" trait with a single impl:
```
trait Post<'a, B> {
    //     ^^^^ these are the "captured" parameters, per RFC 1951
    type Output: Future + 'a;
}

impl<'a, B> Post for () {
    default type Output = /* the hidden type that compiler infers */;
}
```
and the function were then declared like so:
```
fn post<'a, B>(&'a self, _body: &B) -> <() as Post>::Output { .. }
```
Why does this matter?

Look at the next function, login:
```
fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a { .. }
```
this function winds up inferring that the hidden type X is <() as Post<'0, B>>::Output for some fresh inference variable '0, and hence we must infer that X: Future and X: 'a. Since the Output is declared as default in the impl, we can't fully normalize it, and must instead rely on the declared bounds (type Output: Future + 'a, where the 'a here will be the variable '0).

The bug report here seems correct: we should be able to prove that X: 'a by adding the requirement that '0: 'a and not requiring B: 'a. And if you try to write out the model I wrote above, you'll find that the compiler does so (at least I expect you will).

How does this work for associated types?

The compiler handles similar problems already for associated types. The rules were outlined in RFC 1214, along with some of the challenges. I think we should probably be able to apply the compiler's existing heuristics to this problem, but it might take a bit of work.

The relevant function is here:

https://github.com/rust-lang/rust/blob/daa53a52a2667533d5fe59bfcc5b8614b79c3d31/src/librustc/infer/outlives/obligations.rs#L357-L362

In particular, I believe this heuristic is the one that will help:

https://github.com/rust-lang/rust/blob/daa53a52a2667533d5fe59bfcc5b8614b79c3d31/src/librustc/infer/outlives/obligations.rs#L445-L459
Niko Matsakis at 2019-01-17 14:50:27
this stackoverflow question seems related + a funny error message from the compiler about the lifetime, that could be changed?!
chpio at 2019-06-20 10:33:14

A smaller test case from the forum (fails to compile in 1.39-nightly):

fn foo<T>(_: T) -> impl Iterator<Item = i32> + 'static {
    vec![2].into_iter()
}

fn bar() {
    let input = vec![1];
    let output0 = foo(&input); //output0 should have nothing to do with input
    let output1 = foo(input);
}

Kornel at 2019-09-11 12:01:26

Updating the workaround I mentioned with the new syntax for trait aliases:
- type PostFuture<'a> = impl Future + 'a; fn post<'a, B>(&'a self, _body: &B) -> PostFuture<'a> {
- type SomeTr = impl Tr; fn foo<T>(_t: T) -> SomeTr {
Arnav Singh at 2019-09-11 20:00:49
Smallest test case I've managed to produce: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=cf4582e81175a18eac9e2abc39c725a2
```
fn foo<T>(_: T) -> impl Send { }
fn bar<'a>(x: &'a ()) -> impl Send { foo(x) }
```
James Kay at 2019-11-21 17:13:38
This should probably be triaged at this point, so we can decide how much effort to devote to it (and how soon). @Centril?
Alexander Regueiro at 2019-11-21 17:19:33
This really sucks for futures. For example, the return value for this won't be 'static at all; it will be whatever lifetime T has.
```
fn send<T: Serialize>(&self, body: &T) -> impl Future<Output=Result<Response, Error>> + 'static {
    let prepared_body: Vec<u8> = serde_json::to_vec(body);
    // We don't need `body` anymore... but rust thinks we do.
    async move {
        let resp = self.client.send(prepared_body?).await?;
        Ok(serde_json::from_slice(&resp)?)
    }
}
```
What's especially confusing is that even though I marked the impl as 'static, when I actually tried to use it as such, the compiler was assigning the return type a non-static lifetime.

At the very least, please add a warning; as far as I can tell, bounding the impl lifetime with 'static is useless because of the other implicit lifetimes it gains.
Alex Parrill at 2020-01-31 00:38:41
Any news on this? Just ran into this again.
Alex Parrill at 2020-09-24 01:36:03
I wrote up a separate issue on potentially improving the diagnostics for this type of thing: https://github.com/rust-lang/rust/issues/78402.
Dirkjan Ochtman at 2020-10-26 15:53:28
I just ran into this rather baffling bug as well. What's the progress on fixing it?
Rua at 2021-09-26 18:03:36
I also encountered this issue recently and would love to see it fixed, as it makes working with async harder.
Simon Adameit at 2021-10-14 09:58:38
tagging with wg-async with hopes that it will be enqueued as a polish issue for a volunteer from that group to work on.

@rustbot label: +wg-async
Felix S Klock II at 2022-03-07 15:25:58
The important thing to understand about this issue is that it doesn't have anything to do with the lifetime bounds. The lifetime bounds work exactly as you would expect that they would. Consider this minimized version of the issue:
```
fn capture<'o, T>(_t: T) -> impl Send + 'o {}
fn outlives<'o, T: 'o>(_t: T) {}

fn test<'o>(x: ()) -> impl Send + 'o {
    outlives::<'o, _>(capture::<'o, &'_ ()>(&x)); // OK.
    capture::<'o, &'_ ()>(&x)
//                        ^^
// ERROR: Borrowed value does not live long enough.
}
```
Playground link

The line labeled "OK" compiles fine. This shows that the opaque type returned by capture does in fact outlive 'o.

However, Rust won't let a value of this type escape up the stack (without type erasure) because there's no way to give the captured lifetime a name that's valid outside of the function.

A full analysis of this subtle issue is available here.
Travis Cross at 2023-09-16 01:39:49
How can I tell Rust that no, the return type does not, in fact, capture any input lifetimes, and is entirely static?
Rua at 2023-10-09 18:48:06
How can I tell Rust that no, the return type does not, in fact, capture any input lifetimes, and is entirely static?

https://github.com/rust-lang/rust/issues/42940#issuecomment-410762919
Arnav Singh at 2023-10-09 19:25:08
How can I tell Rust that no, the return type does not, in fact, capture any input lifetimes, and is entirely static?

#42940 (comment)

Sadly that doesn't work when I try it, it says the feature is unstable.
Rua at 2023-10-10 07:52:36
If it was possible to use TAIT in stable this issue wouldn't still be open.
Arnav Singh at 2023-10-10 15:16:13

Errors

Works

the model

Why does this matter?

How does this work for associated types?