Confusing type error due to strange inferred type for a closure argument
This example:
pub struct Request<'a, 'b: 'a> {
a: &'a (),
b: &'b (),
}
pub trait Handler: Send + Sync + 'static {
fn handle(&self, &mut Request) -> Result<(),()>;
}
impl<F> Handler for F where F: Send + Sync + 'static + Fn(&mut Request) -> Result<(),()> {
fn handle(&self, _: &mut Request) -> Result<(),()> {
unimplemented!()
}
}
fn make_handler(h: &'static Handler) -> Box<Handler> {
Box::new(move |req| h.handle(req))
}
errors with
error[E0271]: type mismatch resolving `for<'r, 'r, 'r> <[closure@<anon>:14:14: 14:38 h:_] as std::ops::FnOnce<(&'r mut Request<'r, 'r>,)>>::Output == std::result::Result<(), ()>`
--> <anon>:14:5
|
14 | Box::new(move |req| h.handle(req))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bound lifetime parameter , found concrete lifetime
|
= note: concrete lifetime that was found is lifetime '_#9r
= note: required because of the requirements on the impl of `Handler` for `[closure@<anon>:14:14: 14:38 h:_]`
= note: required for the cast to the object type `Handler`
Annotating the closure parameter |req: &mut Response| allow the example to compile.
Interesting, annotating with |req: &&mut Response| produces a similarly-structured error, so I believe we're inferring &&mut here (maybe related to autoderef?).
cc @nikomatsakis @eddyb
Jeffrey Seyfried at 2017-04-05 05:42:41
This is a known issue IIRC, but I don't know how to actually find the original issue.
Eduard-Mihai Burtescu at 2017-04-06 18:05:06
Sorry, I've been putting off this tab for days trying to squeeze out some time to write a more detailed comment. But in short the problem is that we do not infer when to "generalize" the region -- in particular, we will infer a type of
&'a mut Handlerfor some'a, but your type demands a type of&'a mut Handlerfor all'a(i.e., the difference betweenfn(&'a mut Handler)andfor<'a> fn (&'a mut Handler). We do not (yet) infer generalizations except when the expected type provides the hint -- now, in this case, I might expect us to be able to trace things through the expected type. We would I think do so if you were returningBox<FnMut()>, but because theFnMut()requirement is "hidden" (this is a trait that happens to be implemented by closures...) we fail to deduce it. I'm pretty sure there's an open issue on this somewhere.Niko Matsakis at 2017-04-12 17:13:45
Just ran into this (or at least I'm pretty sure it's this, otherwise feel free to split this off into a new bug)
Smaller test case:
fn main() { let suitable = |_| true; vec![1,2,3].into_iter().find(suitable); }Workarounds: Inline suitable into find, or annotate suitable with a
&type, i.e.let suitable = |_: &_| true.I think all of these issues are older duplicates
- #36582
- #32600 (which has the type annotation workaround)
- #26937 (which has the inlining workaround)
- #24680
- #22791
Greg Morenz at 2017-04-15 14:22:00
More duplicates, or at least the same underlying issue:
- #22557
- #34892
- #14037
- #30786 (possibly)
We should decide on a single (or two, one for the diagnostic and one for the underlying issue) canonical issue and deduplicate.
Mark Rousskov at 2017-04-15 18:58:15
Leaving the following issues open, since I don't think they're quite duplicates. The rest of the issues linked in the two comments above are about to be closed referencing this issue.
- https://github.com/rust-lang/rust/issues/36582
- https://github.com/rust-lang/rust/issues/22557
- https://github.com/rust-lang/rust/issues/30786
Mark Rousskov at 2017-04-29 00:33:56
I wanted to re-include and extend @shepmaster's example from #26937 as it includes a potential work-around (?).
Namely that including the closure directly into the call of the function vs. binding it avoids the problem entirely and using type annotations can sometimes avoid the problem. Hopefully this is insightful and those more experienced can weigh in more.
https://is.gd/6jjGgX
fn caller<F>(f: F) where F: Fn(&i32) { f(&42); } fn caller_ref_return<F>(f: F) where F: Fn(&i32) -> &i32 { f(&42); } fn main() { // Works caller(|a| println!("{}", a)); // Works let f = |a: &i32| println!("{}", a); caller(f); // Doesn't work (error: type mismatch) let f = |a| println!("{}", a); caller(f); // Works caller_ref_return(|a| a); // Doesn't work (error: expected bound lifetime parameter, found concrete lifetime) let f = |a: &i32| a; caller_ref_return(f); // Doesn't work (error: type mismatch) let f = |a| a; caller_ref_return(f); }Ralph Minderhoud at 2017-04-29 16:46:58
Not sure if anyone else pointed this out, but wanted to specifically highlight this part of the error message:
for<'r, 'r, 'r>This is unreadable − all the lifetime parameters are assigned the same name!
Phil Ruffwind at 2017-05-27 23:08:23
Another instance of this was just brought up on
#rust-beginnerswith theregexpcrate:let s = "hello"; let re = regex::Regex::new("[aeiou]").unwrap(); let after = re.replace_all(s, |c| "xxx".to_string()); println!("{} -> {}", s, after);fails with the supremely unhelpful error:
error[E0271]: type mismatch resolving `for<'r, 's> <[closure@src/main.rs:7:35: 7:56] as std::ops::FnOnce<(&'r regex::Captures<'s>,)>>::Output == std::string::String` --> src/main.rs:7:20 | 7 | let after = re.replace_all(s, |c| "xxx".to_string()); | ^^^^^^^^^^^ expected bound lifetime parameter, found concrete lifetime | = note: required because of the requirements on the impl of `regex::Replacer` for `[closure@src/main.rs:7:35: 7:56]` error[E0631]: type mismatch in closure arguments --> src/main.rs:7:20 | 7 | let after = re.replace_all(s, |c| "xxx".to_string()); | ^^^^^^^^^^^ --------------------- found signature of `fn(_) -> _` | | | expected signature of `for<'r, 's> fn(&'r regex::Captures<'s>) -> _` | = note: required because of the requirements on the impl of `regex::Replacer` for `[closure@src/main.rs:7:35: 7:56]` error: aborting due to 2 previous errorsCompiles fine if you use
|c: ®ex::Captures|.Jon Gjengset at 2018-01-14 19:08:58
I'm dumb, but isn't this a lot like the monomorphization restriction in Haskell, but with lifetimes instead of types?
Ram at 2018-01-15 06:38:37
Regarding this example that @jonhoo cited (playground link), the cause is that, because the signature for
replace_allrequires only thatR: Replacer, rather than sayR: FnMut(&Captures) -> String, we don't have the "expected type" for the closure's arguments at the time that we need them. As a result, we fail to infer that the region must be bound in the closure (sorry, jargon).I note this because, in addition to improving the error msg, I'd of course prefer if that example just worked -- but resolving that issue (figuring out bound regions more generally) is a bit of an open question. In principle it may be possible, I haven't thought hard about it, but it'll require a lot of refactoring before we get there.
Anyway, we ought to be able to improve the error message in the meantime I suppose.
Niko Matsakis at 2018-01-17 17:49:27
(I suppose that we might, in this particular case, be able to get smarter about the expected type. There is even an open issue on this, but I can't find it just now.)
Niko Matsakis at 2018-01-17 17:51:30
-
struct My(i32); fn call_with_ref<F>(some_closure: F) -> i32 where F: for<'a> Fn(&'a My) -> &'a i32, { let value = My(0); *some_closure(&value) } fn main() { let _f = |arg: &My| &arg.0; //This doesn't work call_with_ref(_f); //This is ok. //call_with_ref(|arg: &My| &arg.0); }Esteban Kuber at 2018-12-03 21:36:45
Maybe similar issue playground
fn main() { let s = ""; ((&|_| s) as &dyn Fn(&str) -> &str)(s); }But this works
fn main() { let s = ""; let f: &dyn Fn(&str) -> &str = &|_| s; f(s); }And another one
Yang Xudong at 2019-07-26 12:03:46
I'm new to Rust, and this had me pulling my hair out for awhile. I still don't know why the "solution" works.
struct S {} fn filter<P>(predicate: P) where P: Fn(&S) -> bool, { predicate(&S{}); } fn main() { // this works filter(|_s| true); // this also works fn cb1(_s: &S) -> bool { true } filter(cb1); // but this doesn't work let cb2 = |_s| true; filter(cb2); }Andrew at 2019-11-09 03:33:11
@amoffat Generally you should avoid putting closures in variables, if you're only going to pass them to a function, because placing the closure expression nested in the call expression lets the compiler deduce a better signature.
Why the signature can even differ is that your
P: Fn(&S) -> boolbound is really sugar for this bound:P: for<'a> Fn(&'a S) -> bool. When you writefilter(|_s| true), the compiler can pick thatfor<'a>for the closure signature as well, and everything works.(the use of
for<...>here is the mathematical ∀, aka "forall", aka "universal quantification" -for<'a>means "for all possible lifetimes'a", and that's very different in power, from inferring a specific lifetime)Your
fn cb1(_s: &S) -> boolis also sugar, forfn cb1<'a>(_s: &'a S) -> bool, so you get that samefor<'a>aspect ahead of thefilter(cb1).With
cb2, however, the closure starts out with nofor<...>lifetimes, and that aspect can't be inferred back fromfilter(cb2).If rustc were smarter, it could wait to see a use of the closure, before type-checking its body, which would also help a few other cases, but it's tricky. Once you have a closure in a variable it can be used multiple times, and those use might have conflicting needs.
Eduard-Mihai Burtescu at 2019-11-09 03:46:09
@eddyb, I still don't understand :) The fix from the SO answer was the following:
fn filter<'a, P>(predicate: P) where P: Fn(&'a S) -> bool, { predicate(&S {}); }Based on what you said, the code now works because I've narrowed the lifetime from the "forall" to a specific lifetime? I'm still unclear on the logic of why a lifetime is needed at all. Is it because the borrow checker needs to make sure that the predicate is only called on a valid borrow of
S? That doesn't make sense because it can only be called with a valid borrow ofS.... so there is nothing to check.Andrew at 2019-11-09 04:10:24
@amoffat Ah, my bad, I didn't click through to the SO post.
&Tnever exists without a lifetime (i.e. it's always sugar when you don't see a lifetime explicitly written), and the lifetime is used to keep track of all the other references derived from that reference - which in your case is none, but that's only a coincidence, as far as the compiler is concerned.For example, with the
P: Fn(&S) -> boolform, the closure doesn't know how long it can keep that reference around, whereas withP: Fn(&'a S) -> boolfrom your last comment, the caller offiltercan pick the'aand, say, stash the reference from the closure to some outer scope.The only reason
predicate(&S {});even compiles in your last comment is another language feature, which makes references to constant expressions have'staticlifetime.If you wrote
let s = S {}; predicate(&s);that wouldn't work, because'acan be whatever the caller offilterwants it to be (including'static), andsdoesn't survive pastfilterreturning.Eduard-Mihai Burtescu at 2019-11-09 04:20:05
The only reason predicate(&S {}); even compiles in your last comment is another language feature, which makes references to constant expressions have 'static lifetime.
If you wrote let s = S {}; predicate(&s); that wouldn't work, because 'a can be whatever the caller of filter wants it to be (including 'static), and s doesn't survive past filter returning.
@eddyb Ah, I didn't catch that from my example! I think that is making the picture more clear for me.
But I'm still unclear on why the compiler cares about the lifetime of
&'a Sreference in the closure. Are you saying it is because of the possibility that the closure can copy the reference into one of its captured fields? And because of this possibility, the borrow checker needs a guarantee ('a) that any input references to the closure will outlive the closure?I know that closures are represented internally as unique structs, so this is starting to make sense. If a closure's capture field can be assigned a reference to an input parameter, then I can see why the compiler cares that the lifetime of the input parameters outlive the closure itself. The fact that I am not actually capturing anything should allow the compiler to relax its restrictions a bit, but currently it is just being unnecessarily strict. Am I on the right track?
Andrew at 2019-11-09 06:03:10
@amoffat You're right that you could do that with a capture, but you don't even need captures, you could be trying to store a
&'static Sinto athread_local!variable, for example.And little of that is understood by the compiler until borrow-checking runs, and borrow-checking enforces the rules that arise from the inferred types, so information can't flow back into inference.
Eduard-Mihai Burtescu at 2019-11-09 06:20:58
That was helpful @eddyb , thanks for explaining
Andrew at 2019-11-09 06:58:28
I just got bitten by this.
I'm implementing a trait on a Closure:impl MyTrait for for<'a> Fn(&'a T) {}Jezza at 2019-12-03 19:35:48
A related issue shows up only when there are at least two reference arguments, but may be worked around by either annotating types or using helper functions:
trait Foo { fn call(&self, s: &str, t: &str); } impl<F: Fn(&str, &str)> Foo for F { fn call(&self, s: &str, t: &str) { self(s,t); } } fn foo<F : Fn(&str, &str)>(f: F) { f("", ""); } fn foo2<F: Foo>(f: F) { f.call("", ""); } fn make_foo<'a, F: Fn(&str, &str) + 'a>(f: F) -> Box<dyn Foo + 'a> { Box::new(f) } fn foo3(f: Box<dyn Foo>) { f.call("", ""); } fn main() { foo(|x,y| ()); // OK foo2(|x:&_,y:&_| ()); // OK foo2(|x,y| ()); // Error, but works with one argument foo3(Box::new(&|x,y| ())); // Error, even with one argument foo3(make_foo(|x,y| ())); // OK! }Paul Nelson at 2020-05-22 16:26:09
A similar case: This code compiles:
fn main () { let mut fields: Vec<&str> = Vec::new(); let pusher = |a| fields.push(a); }but this code does not:
fn main () { let mut fields: Vec<&str> = Vec::new(); let pusher = |a: &str| fields.push(a); }error: borrowed data cannot be stored outside of its closure --> src/main.rs:3:40 | 2 | let mut fields: Vec<&str> = Vec::new(); | - cannot infer an appropriate lifetime... 3 | let pusher = |a: &str| fields.push(a); | ------ --------- ^ cannot be stored outside of its closure | | | | | borrowed data cannot outlive this closure | ...so that variable is valid at time of its declaration error: aborting due to previous error error: could not compile `playground`.I think the explicit type annotation is causing a higher-ranked region to be inferred, while omitting the type annotation causes a concrete region to be inferred.
However, it's extremely surprising that copy-pasting the type from the definition of
fieldsbreaks the code.Aaron Hill at 2020-05-28 05:32:15
The new nightly rustc outputs some different error messages for the first post. The slightly adapted version …
pub struct Request<'a, 'b: 'a> { a: &'a (), b: &'b (), } pub trait Handler: Send + Sync + 'static { fn handle(&self, a: &mut Request) -> Result<(),()>; } impl<F> Handler for F where F: Send + Sync + 'static + Fn(&mut Request) -> Result<(),()> { fn handle(&self, _: &mut Request) -> Result<(),()> { unimplemented!() } } fn make_handler(h: &'static dyn Handler) -> Box<dyn Handler> { Box::new(move |req| h.handle(req)) }… leads to the following error:
error[E0308]: mismatched types --> src/a.rs:14:5 | 14 | Box::new(move |req| h.handle(req)) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other | = note: expected type `for<'r, 's, 't0> std::ops::Fn<(&'r mut a::Request<'s, 't0>,)>` found type `std::ops::Fn<(&mut a::Request<'_, '_>,)>`Christoph Müller at 2020-06-25 10:53:22
With nightly, the example provided by @gmorenz leads to the following clear bug:
error[E0308]: mismatched types --> src/a.rs:3:29 | 3 | vec![1,2,3].into_iter().find(suitable); | ^^^^ one type is more general than the other | = note: expected type `std::ops::FnOnce<(&i32,)>` found type `std::ops::FnOnce<(&i32,)>`Christoph Müller at 2020-06-25 10:54:21
Assigning
P-mediumas discussed as part of the Prioritization Working Group procedure and removingI-prioritize.Additionally we nominate this longstanding issue to be discussed during the next meeting
apiraino at 2020-10-15 13:35:53
This issue was discussed during today's compiler team meeting
Santiago Pastorino at 2020-11-05 19:45:17
Triage: Current output:
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=456b5353ab48999eb273c83cfc0ae6f7
error[E0308]: mismatched types --> src/lib.rs:14:5 | 14 | Box::new(move |req| h.handle(req)) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other | = note: expected type `for<'r, 's, 't0> Fn<(&'r mut Request<'s, 't0>,)>` found type `Fn<(&mut Request<'_, '_>,)>` note: this closure does not fulfill the lifetime requirements --> src/lib.rs:14:14 | 14 | Box::new(move |req| h.handle(req)) | ^^^^^^^^^^^^^^^^^^^^^^^^https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=8342936d11149e38cafc35da516274cb
error[E0521]: borrowed data escapes outside of closure --> src/main.rs:3:28 | 2 | let mut fields: Vec<&str> = Vec::new(); | ---------- `fields` declared here, outside of the closure body 3 | let pusher = |a: &str| fields.push(a); | - ^^^^^^^^^^^^^^ `a` escapes the closure body here | | | `a` is a reference that is only valid in the closure bodyhttps://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=28594d6b627049e7a1de6cd590dfcae8
error[E0308]: mismatched types --> src/main.rs:30:5 | 30 | foo2(|x,y| ()); // Error, but works with one argument | ^^^^ lifetime mismatch | = note: expected type `for<'r, 's> Fn<(&'r str, &'s str)>` found type `Fn<(&str, &str)>` note: this closure does not fulfill the lifetime requirements --> src/main.rs:30:10 | 30 | foo2(|x,y| ()); // Error, but works with one argument | ^^^^^^^^ note: the lifetime requirement is introduced here --> src/main.rs:15:12 | 15 | fn foo2<F: Foo>(f: F) { | ^^^ error: implementation of `Fn` is not general enough --> src/main.rs:31:10 | 31 | foo3(Box::new(&|x,y| ())); // Error, even with one argument | ^^^^^^^^^^^^^^^^^^^ implementation of `Fn` is not general enough | = note: closure with signature `fn(&'2 str, &str)` must implement `Fn<(&'1 str, &str)>`, for any lifetime `'1`... = note: ...but it actually implements `Fn<(&'2 str, &str)>`, for some specific lifetime `'2` error: implementation of `Fn` is not general enough --> src/main.rs:31:10 | 31 | foo3(Box::new(&|x,y| ())); // Error, even with one argument | ^^^^^^^^^^^^^^^^^^^ implementation of `Fn` is not general enough | = note: closure with signature `fn(&str, &'2 str)` must implement `Fn<(&str, &'1 str)>`, for any lifetime `'1`... = note: ...but it actually implements `Fn<(&str, &'2 str)>`, for some specific lifetime `'2`https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0254e96af9155a0a99880e06b4a75b95
error: implementation of `FnOnce` is not general enough --> src/main.rs:3:29 | 3 | vec![1,2,3].into_iter().find(suitable); | ^^^^ implementation of `FnOnce` is not general enough | = note: closure with signature `fn(&'2 i32) -> bool` must implement `FnOnce<(&'1 i32,)>`, for any lifetime `'1`... = note: ...but it actually implements `FnOnce<(&'2 i32,)>`, for some specific lifetime `'2`Esteban Kuber at 2021-09-15 17:15:38
Related: https://github.com/rust-lang/rust/issues/91966
Aaron Hill at 2021-12-29 23:53:05
I don't think any of the posts here mention a workaround (until https://rust-lang.github.io/rfcs/3216-closure-lifetime-binder.html gets implemented).
I have been able to use the following workaround (shown for the example posted by @estebank).
struct My(i32); fn call_with_ref<F>(some_closure: F) -> i32 where F: for<'a> Fn(&'a My) -> &'a i32, { let value = My(0); *some_closure(&value) } fn coerce<F>(closure: F) -> F where F: for<'a> Fn(&'a My) -> &'a i32 { return closure } fn fun() { // Does not work // let _f = |arg: &My| &arg.0; // call_with_ref(_f); // Workaround let _f = coerce(|arg: &My| &arg.0); call_with_ref(_f); // This is ok. call_with_ref(|arg: &My| &arg.0); }While in the above example, the closure can be passed to
call_with_refdirectly also and the problem seems contrived, this is not the case in the below example. Here the explicitcoercefunction seems helpful. I am guessing this will also work withBoxinstead ofOption(for the request handling example in the issue).fn optional_call_with_ref<F>(some_closure: Option<F>) -> i32 where F: for<'a> Fn(&'a My) -> &'a i32, { let value = My(0); *(some_closure.unwrap())(&value) } fn fun2() { // Does not work, have to use explicit coercion. // optional_call_with_ref(Some(|arg: &My| &arg.0)); // Workaround let _f = coerce(|arg: &My| &arg.0); optional_call_with_ref(Some(_f)); }https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=893482858c3f6e96f464b052e1cf321d
I had three questions about this workaround:
- Is it possible to give a name to the trait bound
for<'a> Fn(&'a My) -> &'a i32so that multiple instances of this bound can use the name instead of repeating the definition. - Is it possible to write a macro to generate the
coercefunction? Can macros accept a trait bound and a closure. E.g., to replace withlet _f = coerce_macro!(for<'a> Fn(&'a My) -> &'a i32, |arg: &My| &arg.0) - Is there a better workaround?
Disclaimer: I am fairly new to rust.
Anup Agarwal at 2023-11-02 19:44:04
- Is it possible to give a name to the trait bound
@108anup there's no way of being explicit yet in stable Rust, but nightly Rust supports specifying the
forlifetimelet _f = for<'a> |arg: &'a My| -> &'a i32 { &arg.0 };. Your work-around is what I'd use in stable.Esteban Kuber at 2023-11-03 00:16:20
@108anup: Yes to 1. and 2.: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0a7258b5644e37aca7a25008d5ebca3d
We can't properly give a name to the bound AFAIK since trait alias are unstable, but we can make a coercion function with a user-chosen name which is basically as good in this case. Then the user can just reuse that coercion function without having to repeat the details.
I hope the
for<'a>|&'a ...|syntax gets stabilized soon,nom(or I suppose any closure-heavy lib) is a very special kind of hell without that.Douglas Raillard at 2023-11-08 19:36:14
@douglas-raillard-arm Thanks, this is very helpful!!
I wanted to give a name to the trait bound so that I can use the name in (1) the coercion function (when creating the closure) as well as (2) the function (or struct) that consumes the closure. E.g., in your playground link, the function signature is repeated on line 34. Is there a way to avoid repeating the definition (function signature).
My current workaround has been on the following lines (i.e., store reference to trait object). I think there were some cases of closures I wan't able to represent this way. I don't recall exactly what though.
type SetRealValueFn<'a, 'ctx> = &'a dyn Fn(&'ctx z3::Context, RealNumRep) -> z3::ast::Bool<'ctx>; type GetFeasibleRealValueFn<'a> = &'a dyn Fn(&z3::Solver) -> RealNumRep; type RealNumInt = i32; type RealNumRep = Ratio<RealNumInt>; struct BinarySearchDetails<'a, 'ctx> { // https://stackoverflow.com/questions/27831944/how-do-i-store-a-closure-in-a-struct-in-rust name: &'a String, feasible: Option<RealNumRep>, lo: RealNumRep, hi: RealNumRep, set_value: SetRealValueFn<'a, 'ctx>, get_feasible_closure: GetFeasibleRealValueFn<'a>, use_int: bool, }Anup Agarwal at 2023-11-08 20:39:36
In unstable Rust you can with trait_alias feature: https://doc.rust-lang.org/beta/unstable-book/language-features/trait-alias.html
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=df05613e03998b6cf6a55824b48c1d89
Douglas Raillard at 2023-11-09 10:16:06