`COUNT` constant in `join*!` macros can collide with user's constant used in futures

[robinhundt]

Version

1.46

Platform
Linux nixos 6.12.45 #1-NixOS SMP PREEMPT_DYNAMIC Thu Sep 4 13:31:56 UTC 2025 x86_64 GNU/Linux

Description

The commit 8e999e3#diff-71d8f976620bd493969f9c0ecf5db22825ba06ca05acdcce17f400c5c6ae2c11 changed the scope of the COUNT constant of the join! and try_join! macros so that it is in the same scope as the expansion of the futures expressions $e. If a user of these macros has their own COUNT constant in scope, this leads to a compile error due to mismatching types (usize and u32 in my case) or in the worst case to a change in behavior if the types are compatible but the values are different (repro).

Apparently macros are not hygienic for items, which includes consts... :(

I tried this code:

In one of my projects I have a few test cases which look like this:

    #[tokio::test]
    async fn test_extension() {
        let _g = init_tracing();
        const COUNT: usize = 2 * DEFAULT_OT_BATCH_SIZE;
        let (c1, c2) = local_conn().await.unwrap();
        let rng1 = StdRng::seed_from_u64(42);
        let mut rng2 = StdRng::seed_from_u64(24);
        let choices = random_choices(COUNT, &mut rng2);
        let mut sender = SemiHonestOtExtensionSender::new_with_rng(c1, rng1);
        let mut receiver = SemiHonestOtExtensionReceiver::new_with_rng(c2, rng2);
        let (send_ots, recv_ots) =
            tokio::try_join!(sender.send(COUNT), receiver.receive(&choices)).unwrap();
        for ((r, s), c) in recv_ots.into_iter().zip(send_ots).zip(choices) {
            assert_eq!(r, s[c.unwrap_u8() as usize]);
        }
    }

source

I expected to see this happen:

The code should still compile after a minor version upgrade from 1.45 to 1.46.

Instead, this happened:

This code used to work but unexpectedly broke when upgrading Tokio from 1.45 to 1.46 (or later) with the following error:

error[E0308]: mismatched types
   --> cryprot-ot/src/extension.rs:809:42
    |
809 |             tokio::try_join!(sender.send(COUNT), receiver.receive(&choices)).unwrap();
    |                                     ---- ^^^^^ expected `usize`, found `u32`
    |                                     |
    |                                     arguments to this method are incorrect
    |

This confused me, as clearly the COUNT const is declared as usize and not u32. As explained above, this is due to the scope change of the COUNT constant in the join! and try_join! macros.

[ADD-SP]

@jlizen What do you think?

[jlizen]

From what I recall, it's not trivial to move the constant back into the poll_fn scope because we are relying on the access in broader scope to initialize the non-biased rotator.

But, this is indeed clear semver breakage, sorry to introduce these hygiene concerns.

Perhaps a decent middle ground might be to just make the constant something that is very unlikely to collide with existing user declarations? For instance TOKIO_TRY_JOIN_BRANCH_COUNT?

[ADD-SP]

I think we can use a very very looooooooong and unusual name for this const COUNT: u32, while this is still possible to collide, but very unlikely.

Another solution is storing an u32 into the Rotator.

I prefer the first one as it is the minimal change.

I prefer the second one as the any collision might change the runtime behavior. What do you think @jlizen ?

[jlizen]

I prefer the first option of an unlikely to collide name.

Storing the count inside the rotator would increase the memory size of the future beyond its original (pre-biased feature) footprint since we still need access to the count inside the poll loop as well, resulting in the variable being stored in two places. The current implementation avoids this by using a const generic.

Avoiding additional memory footprint was a chief concern that drove the current implementation.

It seems like a better tradeoff for most users (or presumably all, if a weird enough variable name were picked) to optimize for memory footprint. This came up in this issue since we are instead using an extremely common variable name.

That said I'm open to going with option 2 (store in rotator) if that is the general preference though i'd suggest getting Alice's opinion as well as she guided the initial implementation.

[ADD-SP]

It seems like a better tradeoff for most users (or presumably all

That makes a lot of sense, and I get your point.

But I still think reliability matters more than performance—those very rare bugs can be really painful when they happen.

@Darksonn What do you think?

[jlizen]

Edit: nvm, that won't work, since I still will need to manipulate the variable while preserving its original value. Really would need the const generic to avoid the memory increase. And my understanding is that declaring the const is all is what is breaking hygiene.

I could possibly finagle it to just bundle the counter used inside the poll function inside the rotator. That would trade a bit of complexity for no additional memory footprint. Let me poke around and see what I can do

Would you want a quickie PR to obfuscate the variable name in the meantime as a stopgap for this issue? Or hold out for the deeper fix?

[ADD-SP]

I have another idea, will this work?

#[doc(hidden)]
/// DO NOT USE IT UNLESS YOU ARE DEVELOPING THE TOKIO ITSELF
pub struct PrivU32(u32);

#[derive(Default, Debug)]
pub struct Rotator<const COUNT: PrivU32> {
    next: u32,
}

[jlizen]

Wouldn't that cause the same name collision, just now with a different type from the macro's constant?

[ADD-SP]

Emm, it doesn't work. I thought this approach can at least report the type mismatch error instead of shadowing the variable secretly.

error: `PrivU32` is forbidden as the type of a const generic parameter
   --> tokio/src/macros/join.rs:231:33
    |
231 | pub struct Rotator<const COUNT: PrivU32> {
    |                                 ^^^^^^^
    |
    = note: the only supported types are integers, `bool`, and `char`

[robinhundt]

How about something like this approach?

trait RotatorSelect {
    type Rotator<const COUNT: u32>: Default;
}

struct SelectNormal;
struct SelectBiased;

#[derive(Default)]
struct Rotator<const COUNT: u32>;
#[derive(Default)]
struct BiasedRotator;

impl RotatorSelect for SelectNormal {
    type Rotator<const COUNT: u32> = Rotator<COUNT>;
}

impl RotatorSelect for SelectBiased {
    type Rotator<const COUNT: u32> = BiasedRotator;
}

macro_rules! join {
    (rotator=$rotator:ty, $($total:tt)*) => {
        let rotator = <$rotator as RotatorSelect>::Rotator::<{$($total)*}>::default();
    }
}

fn foo() {
    join!(rotator = SelectNormal, 0 + 1 + 1 + 1);
}

playground

This is a bit more complicated, but this way you don't need to refer to a constant when providing the rotator type.

[jlizen]

How about something like this approach?

Good suggestion, thanks. I think that will work && address all concerns. Poking around with it now.