opam update fails on Alpine Linux

[kit-ty-kate]

Currently the ARM64 CI is failing on alpine while trying to update the repository (from master):
https://arm64.ci.ocamllabs.io/log/saved/docker-build-43ecc3c0743da0075a22689086fef2d7/9ab7d8a55c9c1ddc6aae400f6bd247765c4825e9

The failure is reproducible using the following docker file/steps:

FROM ocaml/opam2:alpine-3.6-ocaml-4.05.0
RUN git pull origin master
RUN opam update

However when using the same commands with the debian image (ocaml/opam2:alpine-3.6-ocaml-4.05.0) it works perfectly.
Both images have the same opam version: 2.0.0~rc (0a141dd) and are starting with the same repository (ocaml/opam-repository#ecba783e1dde04c930a202491aab8f2390b56fd3)

I'm not sure what is different here but maybe you have an idea ?

[AltGr]

It's quite weird: this

[WARNING] Could not read file /home/opam/.opam/repo/default/packages/ocaml-base-compiler/ocaml-base-compiler.3.09.1/opam. skipping:
          In /home/opam/.opam/repo/default/packages/ocaml-base-compiler/ocaml-base-compiler.3.09.1/opam:
          unsupported or missing file format version

shows opam (1.2 if I understand correctly) looking for a file that would only exist on an opam 2 repository (the file would have opam-version: "2.0", which causes the error). The repository commits referred to look clean though. But the messages below:

=-=- Upgrading repositories from older opam format =-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[NOTE] Repository at git+file:///home/opam/opam-repository doesn't define its version, assuming it's 1.2.
Upgrading repository "default"...
Compiler 3.07 successfully converted to package ocaml-base-compiler.3.07

are specific to opam 2 and show that it's converting the repository (note: ocaml/opam-repository#11447 should fix the [NOTE]).

So I don't really understand what's going on. Where is the Dockerfile for the container ? Info on https://hub.docker.com/r/ocaml/opam2/ is very limited.

[kit-ty-kate]

So I don't really understand what's going on. Where is the Dockerfile for the container ? Info on https://hub.docker.com/r/ocaml/opam2/ is very limited.

I know. I have no idea myself. It's all maintained by @avsm

shows opam (1.2 if I understand correctly)

Mmh I'm not sure to understand what you are saying. As said above it's all opam 2.0.0~rc but using opam-repository master (1.2) as default remote, so it needs to convert between the two formats.

[kit-ty-kate]

I got my hands on the way the docker images are built.
Here is an Dockerfile with which I can reproduce the problem:

# Autogenerated by OCaml-Dockerfile scripts
FROM alpine:3.6
LABEL distro_style="apk"
RUN apk update && apk upgrade && \
  apk add build-base bzip2 git tar curl ca-certificates && \
  git clone -b master git://github.com/ocaml/opam /tmp/opam && \
  sh -c "cd /tmp/opam && make cold && mkdir -p /usr/local/bin && cp /tmp/opam/opam /usr/local/bin/opam && cp /tmp/opam/opam-installer /usr/local/bin/opam-installer && chmod a+x /usr/local/bin/opam /usr/local/bin/opam-installer && mkdir -p /usr/local/share/opam && cp shell/wrap-build.sh /usr/local/share/opam && echo 'wrap-build-commands: \"/usr/local/share/opam/wrap-build.sh\"' >> /etc/opamrc.userns && cp shell/wrap-install.sh /usr/local/share/opam && echo 'wrap-install-commands: \"/usr/local/share/opam/wrap-install.sh\"' >> /etc/opamrc.userns && cp shell/wrap-remove.sh /usr/local/share/opam && echo 'wrap-remove-commands: \"/usr/local/share/opam/wrap-remove.sh\"' >> /etc/opamrc.userns && rm -rf /tmp/opam" && \
  strip /usr/local/bin/opam*
FROM alpine:3.6
COPY --from=0 /usr/local/bin/opam /usr/bin/opam
COPY --from=0 /usr/local/bin/opam-installer /usr/bin/opam-installer
RUN apk update && apk upgrade && \
  apk add build-base tar ca-certificates git rsync curl sudo bash libx11-dev nano ncurses-dev && \
  adduser -S -u 1000 opam && \
  echo 'opam ALL=(ALL:ALL) NOPASSWD:ALL' > /etc/sudoers.d/opam && \
  chmod 440 /etc/sudoers.d/opam && \
  chown root:root /etc/sudoers.d/opam && \
  sed -i.bak 's/^Defaults.*requiretty//g' /etc/sudoers
USER opam
WORKDIR /home/opam
RUN mkdir .ssh && \
  chmod 700 .ssh && \
  git config --global user.email "docker@example.com" && \
  git config --global user.name "Docker" && \
  git clone git://github.com/ocaml/opam-repository /home/opam/opam-repository
RUN git -C /home/opam/opam-repository reset --hard 70ca39232f && \
  opam init -a /home/opam/opam-repository && \
  git -C /home/opam/opam-repository pull origin master && \
  opam update

The last RUN instruction I haven't tested formally but it's basically what I did within the console.

[kit-ty-kate]

Any updates on this ?

[AltGr]

Not sure exactly what was going on, but opam update now requires patch. After getting a related error and adding that to the list of apk add packages, the script passed.

[kit-ty-kate]

wow ! This is really weird because #3217 tells us that only patch -p1 -i file is used. I guess somehow BusyBox's patch must be broken for some patches :/

[AltGr]

This may be a huge patch: the way the new opam update works on repositories is to:

• download to some quarantine directory
• compute a patch from the last validated state
• validate that patch based on the known good state (if configured, this is for end-to-end package signing)
• apply the patch

For http repositories, we compute the patch using diff -ruaN, while if on git, we extract it using git. Maybe it's actually the diff command that is failing ?

[avsm]

We do already install GNU patch in the Alpine opam2 base image I think

[rjbou]

@asvm @kit-ty-kate Is there changes in the dockefile above, except for sandboxing?

[avsm]

I think this is fixed thanks to @kit-ty-kate in ocaml-dockerfile: ocurrent/ocaml-dockerfile#8

[rjbou]

ok thanks!