Sys_error during "opam install dune" from within a "dune exec"

[mato]

While upgrading my Mirage/Solo5 E2E test environments from dune 1.8.2 to 1.9.1, opam install dune started failing with a Sys_error exception. I've managed to extract a minimal reproducible test for this error, here:

https://github.com/mato/dune-1.9.1-issue

To reproduce, with an OPAM switch with dune and shexp installed:

git clone https://github.com/mato/dune-1.9.1-issue.git
cd dune-1.9.1-issue
dune exec bin/main.exe

Results in:

[ERROR] The compilation of dune failed at
        "/home/mato/.opam/opam-init/hooks/sandbox.sh build ./boot.exe --release
        -j 3".

#=== ERROR while compiling dune.1.9.1 =========================================#
# context     2.0.3 | linux/x86_64 | ocaml-base-compiler.4.07.1 | https://opam.ocaml.org#4b9884e4
# path        /tmp/dune-1.9.1-issue/switch/_opam/.opam-switch/build/dune.1.9.1
# command     ~/.opam/opam-init/hooks/sandbox.sh build ./boot.exe --release -j 3
# exit-code   1
# env-file    ~/.opam/log/dune-23705-6d07ae.env
# output-file ~/.opam/log/dune-23705-6d07ae.out

and the entire error from output-file is:

Error: exception Sys_error("/tmp/dune-1.9.1-issue/_build/install/default/lib: No such file or directory")
Backtrace:
Raised by primitive operation at file "src/stdune/path.ml", line 878, characters 25-50
Called from file "src/findlib.ml", line 441, characters 6-31
Called from file "list.ml", line 99, characters 22-25
Called from file "src/stdune/list.ml", line 5, characters 19-33
Called from file "src/stdune/list.ml", line 39, characters 29-39
Called from file "src/findlib.ml", line 440, characters 4-265
Called from file "src/findlib.ml", line 453, characters 20-37
Called from file "src/findlib.ml", line 457, characters 2-21
Called from file "src/lib.ml", line 1334, characters 6-34
Called from file "src/super_context.ml", line 288, characters 4-70
Called from file "src/gen_rules.ml", line 327, characters 6-161
Called from file "src/fiber/fiber.ml", line 99, characters 6-13

I think this is some interaction between opam install being called from within a dune exec. I'm not including the .env file here as this is a public forum, but the above steps are 100% reproducible for me.

Thanks for the report. I was able to locate the issue with the backtrace.

[mato]

Great. Is there anything I can use as a workaround until a new release is out? I'm guessing the list of directories is coming from something in the environment that the top-level dune exec is setting?

You can do a manual mkdir -p _build/default/lib just before the call to dune exec. Yh, that something like that. Although, I didn't expect this code to be triggered during normal dune operations as it is the code that's loading the whole findlib db. It seems to be a side-effect of library variants. I'm checking with other dune devs how we can change that

[mato]

Ok, now I'm confused. I added a mkdir -p _build/default/lib to the test driver and all seemed fine. Then I run it from within the CI environment and it runs into the same problem for _build/install/default/lib. So I added that too (mato/e2e-mirage-solo5@f1a7b04), yet installing dune still fails as if that directory didn't exist:

$ cat ~/.opam/log/dune-11984-b22031.out
Error: exception Sys_error("/tmp/e2e/_build/install/default/lib: No such file or directory")
Backtrace:
Raised by primitive operation at file "src/stdune/path.ml", line 878, characters 25-50
Called from file "src/findlib.ml", line 441, characters 6-31
Called from file "list.ml", line 99, characters 22-25
Called from file "src/stdune/list.ml", line 5, characters 19-33
Called from file "src/stdune/list.ml", line 39, characters 29-39
Called from file "src/findlib.ml", line 440, characters 4-265
Called from file "src/findlib.ml", line 453, characters 20-37
Called from file "src/findlib.ml", line 457, characters 2-21
Called from file "src/lib.ml", line 1334, characters 6-34
Called from file "src/super_context.ml", line 288, characters 4-70
Called from file "src/gen_rules.ml", line 327, characters 6-161
Called from file "src/fiber/fiber.ml", line 99, characters 6-13

$ ls -ld /tmp/e2e/_build/install/default/lib
drwxr-xr-x 2 build build 6 Apr 30 11:16 /tmp/e2e/_build/install/default/lib

Here's the corresponding .env with some irrelevant parts redacted:

CAML_LD_LIBRARY_PATH=/tmp/e2e/run/switch/_opam/lib/stublibs:/tmp/e2e/run/switch/_opam/lib/ocaml/stublibs:/tmp/e2e/run/switch/_opam/lib/ocaml
CDPATH=
DUNE_CONFIGURATOR=/home/build/.opam/default/bin/ocamlc.opt
HOME=/home/build
INSIDE_DUNE=1
LOGNAME=build
MAKEFLAGS=
MAKELEVEL=
MANPATH=:/tmp/e2e/_build/install/default/bin:/tmp/e2e/run/switch/_opam/man
OCAMLFIND_IGNORE_DUPS_IN=/tmp/e2e/_build/install/default/lib
OCAMLPATH=/tmp/e2e/_build/install/default/lib
OCAML_TOPLEVEL_PATH=/tmp/e2e/run/switch/_opam/lib/toplevel
OLDPWD=/tmp/solo5-aefe44
OPAMNOENVNOTICE=true
OPAMSWITCH=/tmp/e2e/run/switch
OPAMYES=true
OPAM_PACKAGE_NAME=dune
OPAM_PACKAGE_VERSION=1.9.1
OPAM_SWITCH_PREFIX=/tmp/e2e/run/switch/_opam
PATH=/tmp/e2e/run/switch/_opam/bin:/tmp/e2e/_build/install/default/bin:/home/build/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/build/node_modules/.bin
PWD=/tmp/e2e
SHELL=/bin/bash
SHLVL=1
SURF_ARTIFACT_DIR=/tmp/solo5-aefe44/surf-artifacts
SURF_BUILD_TYPE=e2e
SURF_NWO=Solo5/solo5
SURF_ORIGINAL_TMPDIR=/tmp
SURF_REPO=https://github.com/Solo5/solo5
SURF_SHA1=aefe443d79bcc6f440980d502a78e8355306f6d6
TEMP=/tmp/t-solo5-aefe44
TMP=/tmp/t-solo5-aefe44
TMPDIR=/tmp/t-solo5-aefe44
USER=build