Promote some failing tests?

[lpw25]

For me, on a clean checkout, these tests fail and need promoting. Chris also saw the same, but Luke didn't. The CI doesn't seem to be failing, so presumably it doesn't either.

I made this PR so others can try and figure out what is going on. It should only be merged if these promotions are in fact correct.

[ccasin]

@goldfirere could you nominate a relevant with-kinds person to investigate these failures? Our best guess at the moment is that there's some randomness that depends on your local environment, as they didn't fail in CI but are failing locally.

[ccasin]

(@liam923 is going to take a look)

[liam923]

I've been unable to reproduce this failure. I've tried running on different boxes under different configurations. How confident are you that the build was in fact clean? The test failures are all consistent with a stale version of the stdlib being used - specifically, one where the stdlib was compiled without the -infer-with-bounds flag.

[ccasin]

I've been unable to reproduce this failure. I've tried running on different boxes under different configurations. How confident are you that the build was in fact clean? The test failures are all consistent with a stale version of the stdlib being used - specifically, one where the stdlib was compiled without the -infer-with-bounds flag.

Yes, we're confident we cleaned. I just tested again with the following steps:

$ rm -rf fresh
$ git clone git@github.com:ocaml-flambda/flambda-backend.git fresh
$ cd fresh
$ autoconf27 && ./configure --enable-dev --enable-ocamltest --prefix="$(pwd)"/_install
$ make runtest-upstream

And I get a bunch of failures:

List of failed tests:
    tests/typing-jkind-bounds/'composite.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/'predef.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/'printing.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/'records.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/'variants.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/'with_basics.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/no-infer-across-modules/'test.ml' with line 8 (expect) 
    tests/typing-jkind-bounds/subsumption/'basics.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/subsumption/'constraint.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/subsumption/'functors.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/subsumption/'modalities.ml' with line 3 (expect) 
    tests/typing-jkind-bounds/subsumption/'recursive.ml' with line 3 (expect) 
    tests/typing-modal-kinds/'basics.ml' with line 3 (expect)

I will slack you some additional details.

[liam923]

Okay I've figured out what's going on. The test failures are consistent with the stdlib not getting passed the -infer-with-bounds. Until dune 3.7.1, dune incorrectly passed flags to the stdlib - see ocaml/dune#7241. @ccasin was building with 3.7.0, whereas I was building with 3.9.0. Presumably @lpw25 also used a dune < 3.7.1.

Are we okay with bumping the required dune version to >= 3.7.1?

[ccasin]

Thanks for figuring this out, @liam923! I think it would be fine to require such a dune. @mshinwell ?

[mshinwell]

I think it might be better to introduce a file-level attribute for this behaviour, to avoid having to bump everyone's dune.

[goldfirere]

I think it's both fine to require bumping everyone's dune and also to change this to an attribute. I feel like we thought about using an attribute for this instead of a flag -- at least I hope we did. My experience from Haskell (which is too configurable) is that it's best to have type-checking-dependent choices in the file itself. And if it's an attribute, we can have e.g. [@@@infer_with_bounds] turn it on, [@@@infer_with_bounds ~false] turn it off (it's like there's an optional parameter that defaults to ~true), and [@@infer_with_bounds] do it for just one declaration (note only two @-signs).

But that would take a little time to engineer. I think, for now, we should just ask folks to upgrade their dune.