`@allocated` behavior changed in 1.12

[Moelf]

1.11

julia> @allocated a2 = rand(3);

julia> a2
3-element Vector{Float64}:
 0.52767349525505
 0.2269015858381862
 0.17957399950312425

1.12

julia> @allocated a2 = rand(3);

julia> a2
ERROR: UndefVarError: `a2` not defined in `Main`
Suggestion: check for spelling errors or missing imports.

[serenity4]

#58057 changed from evaluating the expression as it is to wrapping it with Base.allocated, emitting a closure when the expression is not a simple function call. That means the assignment is now put in a closure and has no effect.

It is unclear to me how we would restore the original behavior, besides special-casing top-level assignments (=, +=, etc) because the main objective of the PR was to change the scope. Something like x = 3; @allocated +(1, (x += 2)) seems very cumbersome to support with the new design. Perhaps fixing top-level assignments removes enough breakage that it is fine to theoretically break weird constructs.

cc @vtjnash

[vtjnash]

No, this is very intentional that this changed scope now, as the intent is not to measure the cost of assigning to a global variable, but rather the cost of the expression. I agree it is a bit inconvenient, but @time and @timev are much better for use

[oscardssmith]

IMO this is an unacceptable breakage. I would be very surprised if this didn't break tests for a lot of packages.

[Moelf]

as the intent is not to measure the cost of assigning to a global variable

In my tests, I remember I had to do this because otherwise the compiler would realize the result is not used and optimize something away. I suppose I can always write the tests more carefully but this change in behavior is not expected...

[vtjnash]

You can be surprised if you want to, but we did check pkgeval and already decided this is the best course of action. That said, we strongly advise against testing @allocated numbers since they are unreliable in general

[Moelf]

we strongly advise against testing @ allocated numbers since they are unreliable in general

what would you suggest for testing if broadcasting is fused for my custom, disk-backed array type?

[ufechner7]

This is what I do:

# benchmark loop!
pos, vel, posd, veld = init2()
t = @benchmark KiteModels.loop!($kps4, $pos, $vel, $posd, $veld)
@test t.memory <= 0