[new release] eio_posix, eio_main, eio_luv, eio_linux and eio (0.9)

[talex5]

Eio implementation for POSIX systems

• Project page: https://github.com/ocaml-multicore/eio
• Documentation: https://ocaml-multicore.github.io/eio/

CHANGES:

New features:

• Add eio_posix backend (@talex5 @haesbaert Initial Eio_posix backend ocaml-multicore/eio#448 eio_posix: yield on IO ocaml-multicore/eio#477, reviewed by @avsm @patricoferris @polytypic).
  This replaces eio_luv on all platforms except Windows (which will later switch to its own backend). It is a lot faster, provides access to more modern features (such as openat), and can safely share OS resources between domains.

• Add subprocess support (@patricoferris @talex5 eio_posix: initial support for subprocesses ocaml-multicore/eio#461 Add inherit_fds fork action ocaml-multicore/eio#464 eio_linux: add subprocess support ocaml-multicore/eio#472, reviewed by @haesbaert @avsm).
  This is the low-level API support for eio_linux and eio_posix. A high-level cross-platform API will be added in the next release.

• Add Fiber.fork_seq (@talex5 Add Fiber.fork_seq ocaml-multicore/eio#460, reviewed by @avsm).
  This is a light-weight alternative to using a single-producer, single-consumer, 0-capacity stream, similar to a Python generator function.

Bug fixes:

• eio_linux: make it safe to share FDs across domains (@talex5 eio_linux: make it safe to share FDs across domains ocaml-multicore/eio#440, reviewed by @haesbaert).
  It was previously not safe to share file descriptors between domains because if one domain used an FD just as another was closing it, and the FD got reused, then the original operation could act on the wrong file.

• eio_linux: release uring if Linux is too old (@talex5 eio_linux: release uring if Linux is too old ocaml-multicore/eio#476).
  Avoids a small resource leak.

• eio_linux: improve error handling creating pipes and sockets (@talex5 eio_linux: improve error handling creating pipes and sockets ocaml-multicore/eio#474, spotted by @avsm).
  If we get an error (e.g. too many FDs) then report it to the calling fiber, instead of exiting the event loop.

• eio_linux: wait for uring to finish before exiting (@talex5 eio_linux: wait for uring to finish before exiting ocaml-multicore/eio#470, reviewed by @avsm).
  If the main fiber raised an exception then it was possible to exit while a cancellation operation was still in progress.

• eio_main: make EIO_BACKEND handling more uniform (@talex5 eio_main: make EIO_BACKEND handling more uniform ocaml-multicore/eio#447).
  Previously this environment variable was only used on Linux. Now all platforms check it.

• Tell dune about EIO_BACKEND (@talex5 Tell dune about EIO_BACKEND ocaml-multicore/eio#442).
  If this changes, dune needs to re-run the tests.

• eio_linux: add some missing close-on-execs (@talex5 eio_linux: add some missing close-on-execs ocaml-multicore/eio#441).

• eio_linux: read_exactly fails to update file offset (@talex5 eio_linux: read_exactly fails to update file_offset ocaml-multicore/eio#438).

• Work around dune enabled_if bug on non-Linux systems (@polytypic Work around various opam and dune issues on non-linux systems ocaml-multicore/eio#475, reviewed by @talex5).

• Use raw system call of getrandom for glibc versions before 2.25 (@zenfey use raw system call of getrandom for glibc version before 2.25 ocaml-multicore/eio#482).

Documentation:

• Add HACKING.md with hints for working on Eio (@talex5 Add HACKING.md with hints for working on Eio ocaml-multicore/eio#443, reviewed by @avsm @polytypic).

• Improve worker pool example (@talex5 Improve worker pool example ocaml-multicore/eio#454).

• Add more Conditions documentation (@talex5 Add more Eio.Condition documentation ocaml-multicore/eio#436, reviewed by @haesbaert).
  This adds a discussion of conditions to the README and provides examples using them to handle signals.

• Condition: fix the example in the docstring (@avsm Condition: fix the example in the docstring ocaml-multicore/eio#468).

Performance:

• Add a network benchmark using an HTTP-like protocol (@talex5 Add a network benchmark using an HTTP-like protocol ocaml-multicore/eio#478, reviewed by @avsm @patricoferris).

• Add a benchmark for reading from /dev/zero (@talex5 Add a benchmark for reading from /dev/zero ocaml-multicore/eio#439).

Other changes:

• Add CI for macOS (@talex5 Add CI for macOS ocaml-multicore/eio#452).

• Add tests for pread, pwrite and readdir (@talex5 Add tests for pread, pwrite and readdir ocaml-multicore/eio#451).

• eio_linux: split into multiple files (@talex5 Prepare to split eio_linux into separate files ocaml-multicore/eio#465 eio_linux: split into multiple files ocaml-multicore/eio#466, reviewed by @avsm).

• Update Dockerfile (@talex5 Update Dockerfile ocaml-multicore/eio#471).

• Use dune.3.7.0 (@patricoferris Use dune.3.7.0 ocaml-multicore/eio#457).

• Mint exclusive IDs across domains (@TheLortex Mint exclusive IDs ocaml-multicore/eio#480, reported by @haesbaert, reviewed by @talex5).
  The tracing currently only works with a single domain anyway, but this will change when OCaml 5.1 is released.

[talex5]

There is one CI error, but it looks like a problem with the network:

#=== ERROR while fetching sources for eio.0.9 and eio_posix.0.9 ===============#
OpamSolution.Fetch_fail("https://github.com/ocaml-multicore/eio/releases/download/v0.9/eio-0.9.tbz 
(Curl failed: \"/usr/bin/curl --write-out %{http_code}\\\\n --retry 3 --retry-delay 2 --user-agent opam/2.2.0~alpha~dev -L -o /home/opam/.opam/5.0/.opam-switch/sources/eio.0.9/eio-0.9.tbz.part -- https://github.com/ocaml-multicore/eio/releases/download/v0.9/eio-0.9.tbz\" exited with code 6)")

The URL given works for me. I tried rebuilding and got the same error. Also, the web UI seems very slow today.

[avsm]

curl exit code 6 above is a DNS resolution failure I believe, so that's likely a transient CI issue.

[talex5]

I rebuilt it again and it worked this time.

[kit-ty-kate]

Thanks