Array reinterpret operations

[mshinwell]

This is a different approach from the one I took previously to adding the "array reinterpret" operations, e.g. given an int64# array, get the two values corresponding to the type #(float# * float#) from byte offset 8 (which is unaligned with respect to the product type as a whole). I think this way is easier: the Lambda primitives now mirror what happens in the middle end, namely they take both the array kind and the array ref/set kind. (This means that all this information basically just passes through Lambda_to_flambda, which in fact could now become simpler, as some conversion functions could now be deleted were it not for the float array optimization.)

• The array kind determines indexing
• The array ref/set kind determines what is actually read or written.

We should be able to expose reinterpret primitives in Translprim which map to these existing primitives, just with the kinds appropriately set, I hope.

[ccasin]

This PR is now ready for review.

• @rtjoa: Could you review the typing and lambda bits (and corresponding tests)?
• @TheNumbat: Could you review everything from the middle-end onwards? Please also check if you think the tests of dynamic behavior are sufficient.

[TheNumbat]

Needs tests with mixed unboxed products and reinterpreting non-value data as values/immediates.

[ccasin]

I would like to have these but I think these cases are intentionally not supported by the current implementation and we plan to ship without them, as they aren't needed by my initial use case

Is that right @mshinwell ?

If it actually is supported to have a mixture of unboxed things and immediates in an array I will definitely write tests for that because it would simplify one aspect of the use case

[TheNumbat]

iirc unboxed products currently have to be all-scannable or all-unfriendly, but it didn't look like this pr prohibited loading e.g. #(int * int) from an int64# array? (Also the tests only create tuples where the fields are all the same layout; in any case it should test like #(int64 * float64))

[mshinwell]

Regarding "unboxed things and immediates in an array": I think that should be supported? See ignorable_product_element_kind in lambda.mli.

[mshinwell]

Regarding reinterpreting non-value data as values: I think we should prohibit this, and I have added checks to try to do so.

[mshinwell]

Regarding reinterpreting non-value data as immediates: this is allowed, and the bottom bit is set now upon reading, as for the existing (non-array) reinterpret primitives.

[mshinwell]

So it seems like here we maybe just need tests for "a mixture of unboxed things and immediates in an array"?

[rtjoa]

The changes to typing, lambda, and the testsuite look good to me.

Thinking aloud: there are a few mechanisms that interact with each other in Translprim:

• The glb_array_* functions use the "cheat" that treats Pgenarray* as a super kind of all other array kinds.
• lookup_primitive defaults the array kind for array reinterpret operations to Pgenarray, then specialize_primitive takes the GLB of this kind and the array_kind_of_elt ~elt_sort:None of the relevant part of its type.
• array_kind_of_elt, when elt_sort is None, defaults to calling classify with the sort of its parameter ty
• classify, when called with sort value and a type not found in the environment, returns Any, causing array_kind_of_elt to return Pgenarray.

Squinting at the above makes me wonder if we can accidentally reinterpret-get something as a Pgenarray. I think we can't, but wanted to say this all aloud for those more familiar with this part of the compiler.

• I assume the existing code, which relies on array_kind_of_elt in a similar way, is correct 🙂.
• When I make classify assert false in the "not found" case, all the tests still pass.