Prevent [@unboxable] from applying to local values

[lthls]

The current implementation of unboxing for parameters in Flambda2 will create allocations in the unboxed version, with the hope that these allocations will be simplified away. However, for values with local mode, inserting such allocations is bad, because we don't know the region in which they're going to live.
So until we've found a proper way to handle that, unboxing is disallowed for values with local mode.
Note that an [@unboxable] attribute does not prevent the type-checker from inferring a local mode, so we can end up rejecting an [@unboxable] attribute even though the code contains no explicit local annotations.

[lthls]

Here is an example where the new warning triggers:

module M : sig end = struct
  type 'a t = { a : 'a; b : 'a }
  let f (x[@unboxable]) =
    x.a + x.b

  let g a b =
    let r = f {a; b} in
    r

  let () = assert (g 1 2 = g 2 1)
end

The type-checker infers that x can be local, but unboxing it would introduce local allocations in a function that does not have a region.
I'll add this test to the testsuite after #2321 is merged.

[lthls]

Note: we're thinking of interpreting the [@unboxable] attribute as implying a mode annotation requiring heap allocation.

[lpw25]

What's the plan to make this work with local allocations in the longer term?

[lthls]

We don't have a solution at the moment. If the type-checker understands that local unboxable parameters introduce a local allocation, then we could have the appropriate regions inserted when needed, and proper compilation errors when the new allocation would escape its region (I'm not sure it is actually possible). I think that's the easiest solution, but it requires the help of someone more familiar with the typing part of locals.
Alternatively, we could use a different scheme for unboxable parameter compilation that will only succeed if it does not have to actually insert allocations. If we do that, then it doesn't matter whether the parameter is local or not. But we haven't started work on that at all, so it's not yet clear how hard it would be to do. Getting good error messages for the cases where we cannot do it would likely be quite challenging.

[lpw25]

Would you mind showing me an example of a function before and after the initial [@unboxable] translation? I'd like to understand this issue a bit better. Maybe an example where we aren't currently able to eliminate the boxes too if that's easy to show.

[lthls]

Sure.

(* Before translation *)
let f (x[@unboxable] : local_ (_ * _)) = some_code x

(* After translation *)
let f_unboxed x0 x1 = let x = (x0, x1) in some_code x
let f x = let x0, x1 = x in f_unboxed x0 x1

The new (x0, x1) allocation needs to be a local allocation, as the contents could be locally-allocated themselves, but I'm not sure in which region it should go. Before this PR it goes in the parent's region, but I'm not sure it is safe and it triggers an assertion in Flambda2 (a heap-returning function should not allocate in its caller's region).
The example from my second post triggers the assertion, and is simplified from an actual piece of code in the compiler (which does not have [@unboxable], but we did a test run with systematic unboxing to try to find bugs).

[lpw25]

Thanks. I think we can do a lot better than just disabling it for local parameters, but it does require a little bit of work. Basically, you want to rule out functions like this:

let foo (local_ x) = x

that take a local thing from outside the current region and return it. They would require allocating in the parent's region and that's not a great idea. Such functions have the parameter x typed at the mode "regional" that sits between local and global and means "value that might be on the stack but is not in the current region". Any function which could instead be typed with x at mode local can just have this allocation done inside the region of the function. And obviously any function where x is at mode global is fine.

This means that, when a parameter is marked as [@@unboxable] you want to create an allocation mode that ranges over only global and local (i.e. not "regional"), and then type-check using that. This is similar to your plan to force the mode to global but it will still support most functions that take local parameters too.

[lpw25]

Concretely, when a parameter is annotated as unboxable you want this if to go down the else branch even when region_locked is true. If you do that then it will always be safe to do the unboxing within the function's region (of just in the body of the function if it has no region before optimisation: if the region would get removed during translcore due to lack of local allocations you still actually need it) using the mode of the parameter as the mode of the allocation.

[lthls]

Does it mean that the transformation has to happen before or during translcore ? Doing that after the normal insertion of regions looks a bit fragile.

[lpw25]

It would be best to do the insertion of the allocation during translcore, then there is no danger of it removing the region. Alternatively, you could annotate the region during translcore in some way to say that it can't be removed until after the transformation has been done.