Higher ranked lifetime error when checking auto traits of async functions containing calls to async closures which capture a local

[jdonszelmann]

I tried this code:

#![feature(async_fn_traits)]

use std::ops::AsyncFn;

// this function is important, inlining it breaks the poc.
// We also tried replacing it with `identity` but the fact that this is an
// async function calling the closure inside is also relevant.
async fn do_not_inline(f: impl AsyncFn()) {
    f().await;
}

fn foo() -> impl Send {
    async move {
        // just replacing this with unit, leaves the error with `Send` but fixes the
        // "higher-ranked lifetime error"
        let s = &();

        do_not_inline(async || {
            &s;
        })
        .await;
    }
}

I expected to see this happen: compilation succeeds
Instead, this happened:

error: implementation of `Send` is not general enough
  --> src/lib.rs:13:5
   |
13 | /     async move {
14 | |         // just replacing this with unit, leaves the error with `Send` but fixes the
15 | |         // "higher-ranked lifetime error"
16 | |         let s = &();
...  |
21 | |         .await;
22 | |     }
   | |_____^ implementation of `Send` is not general enough
   |
   = note: `Send` would have to be implemented for the type `&'0 &()`, for any lifetime `'0`...
   = note: ...but `Send` is actually implemented for the type `&'1 &()`, for some specific lifetime `'1`

error: higher-ranked lifetime error
  --> src/lib.rs:13:5
   |
13 | /     async move {
14 | |         // just replacing this with unit, leaves the error with `Send` but fixes the
15 | |         // "higher-ranked lifetime error"
16 | |         let s = &();
...  |
21 | |         .await;
22 | |     }
   | |_____^

error: could not compile `noteslsp` (lib) due to 2 previous errors

try it here: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=ab71ed9cfa6f91320c7a24fe7f2bdca9

note, we got here through a bigger example, the impl Send used to come from the async_trait library where it produced Pin<Box<dyn Future<Output=()> + Send>> but most of that was not relevant for the mcve.

Meta

rustc --version --verbose:

rustc 1.85.0-nightly (d117b7f21 2024-12-31)
binary: rustc
commit-hash: d117b7f211835282b3b177dc64245fff0327c04c
commit-date: 2024-12-31
host: x86_64-unknown-linux-gnu
release: 1.85.0-nightly
LLVM version: 19.1.6

cc: @compiler-errors

@rustbot label +F-async_closure +T-compiler +A-auto-traits

[compiler-errors]

This is likely just a manifestation of #110338.

[jdonszelmann]

unfortunately if you use #[async_trait::async_trait] this setup is very easy to accidentally create. My non minimized example was only 200 lines of not very complicated async code. @WaffleLapkin just said something (in person) I think is worth mentioning too, is that the error for this bubbles up to a confusing place. The usage of the AsyncFn could be anywhere, and is not really mentioned. But the diagnostic (and problem) only comes at the place where the Future gets the impl Send requirement (here foo). Even in my original example, these were 4 functions away from eachother.

[compiler-errors]

Yep, neither of those are special wrt this issue. The many dozens of issues that #110338 links to also are both trivial to trigger and give terrible error messages. The underlying issue is practically impossible to solve at this point though.

[jdonszelmann]

I see :(

[jdonszelmann]

well then I guess this is one for on the same list 🤷

[compiler-errors]

For the record, this doesn't even really have to do with async closures. This is the same bug but written with a manual "async fn" trait and which uses closures that return async blocks:

use std::future::Future;

trait CallAsyncFn {
    type Fut<'a>: Future
    where
        Self: 'a;

    type Output;

    fn call(&self) -> Self::Fut<'_>;
}

impl<F, Fut, T> CallAsyncFn for F
where
    F: Fn() -> Fut,
    Fut: Future<Output = T>,
{
    type Fut<'a>
        = Fut
    where
        Self: 'a;

    type Output = T;

    fn call(&self) -> Self::Fut<'_> {
        self()
    }
}

// this function is important, inlining it breaks the poc.
// We also tried replacing it with `identity` but the fact that this is an
// async function calling the closure inside is also relevant.
async fn do_not_inline(f: impl CallAsyncFn) {
    f.call().await;
}

fn foo() -> impl Send {
    async move {
        // just replacing this with unit, leaves the error with `Send` but fixes the
        // "higher-ranked lifetime error"
        let s = &();

        do_not_inline(|| async {
            &s;
        })
        .await;
    }
}

I'm almost certain that the reason this manifests for AsyncFn (and the trait in the example above) is because it uses a GAT for the future it returns, and there's an instance of problematic associated type normalization involving that GAT that can't be done due to the lifetime erasure that happens inside of coroutines.

[EqualMa]

AsyncFn is only needed when the returned Future is generic over the lifetime of &self (The ref lifetime of the type of the closure it self). In the example, the async closure's returned future is not generic over the lifetime of it self. So we can just use impl Fn<(), Output: Future<Output = ()>>, which avoids naming the type of returned Future and also specified the future Output type. To make that work in stable rust, we need to define a trait alias so that we don't need to name the closure's output type.

use std::future::Future;

// 0 stands for 0 arguments
trait Fn0OutputFuture: Fn() -> Self::OutputFuture {
    type OutputFuture: Future<Output = Self::FutureOutput>;
    type FutureOutput;
}

impl<F: Fn() -> OutputFuture, OutputFuture: Future> Fn0OutputFuture for F {
    type OutputFuture = OutputFuture;
    type FutureOutput = OutputFuture::Output;
}

async fn do_not_inline(f: impl Fn0OutputFuture<FutureOutput = ()>) {
    f().await;
}

fn foo() -> impl Send {
    async move {
        // just replacing this with unit, leaves the error with `Send` but fixes the
        // "higher-ranked lifetime error"
        let s = &();

        do_not_inline(async || {
            &s;
        })
        .await;
    }
}

I come to this issue when I'm trying to find out why rust team decides AsyncFn* is special compared to Fn*. Maybe Fn and FnMut should also make the output type generic over the lifetime of &'a self. Then AsyncFn<Args, Output = T> is just for<'a> Fn<Args, Output<'a>: Future<Output = T>>. The stable syntax of Fn() -> T desugars to for<'a> Fn<(), Output<'a> = T>

[conradludgate]

Even though it's likely the same bug, @\compiler-errors asked me to post this here as another reproduction, as it's potentially easier to understand

async fn run(mut op: impl AsyncFnMut() -> ()) {
    op().await;
}

async fn foo() -> usize {
    let mut x = 0;
    let op = async || { 
        x += 1;
    };
    run(op).await;
    x
}

fn is_send<T: Send>(t: T) -> T { t }
fn check() {
    is_send(foo());
}