Support all operating systems, even hobby OSes, with a "bring your own OS layer" in the std lib

[andrewrk]

The std.Target.Os enum recognizes the following operating systems:

  freestanding
  ananas
  cloudabi
  dragonfly
  freebsd
  fuchsia
  ios
  kfreebsd
  linux
  lv2
  macosx
  netbsd
  openbsd
  solaris
  windows
  haiku
  minix
  rtems
  nacl
  cnk
  aix
  cuda
  nvcl
  amdhsa
  ps4
  elfiamcu
  tvos
  watchos
  mesa3d
  contiki
  amdpal
  hermit
  hurd
  wasi
  emscripten
  zen
  uefi

This list is the list of targets that LLVM supports + zen (a hobby OS not maintained for over 1 year) + UEFI.

It doesn't make sense to put every hobby OS into this list, but it does make sense to support them! It should be possible for people to take advantage of Zig's cross platform abstractions without having to get support for their hobby OS upstreamed into Zig.

I propose to do this with 2 things:

• Add other tag to std.Target.Os
• Support an OS layer struct exposed in the root source file (next to pub fn main())

This allows hobby OS developers to maintain a zig package that makes the zig standard library support their OS. Application developers could use it like this:

pub const os = @import("my_hobby_os_package");

pub fn main() void {
    // ...
}

Next, standard library abstractions will detect when they should utilize this. If the operating system is POSIX compliant, then many things will Just Work. For example, std.os.read is currently defined like this:

/// Returns the number of bytes that were read, which can be less than
/// buf.len. If 0 bytes were read, that means EOF.
/// If the application has a global event loop enabled, EAGAIN is handled
/// via the event loop. Otherwise EAGAIN results in error.WouldBlock.
pub fn read(fd: fd_t, buf: []u8) ReadError!usize {
    if (builtin.os == .windows) {
        return windows.ReadFile(fd, buf);
    }

    if (builtin.os == .wasi and !builtin.link_libc) {
        const iovs = [1]iovec{iovec{
            .iov_base = buf.ptr,
            .iov_len = buf.len,
        }};

        var nread: usize = undefined;
        switch (wasi.fd_read(fd, &iovs, iovs.len, &nread)) {
            0 => return nread,
            else => |err| return unexpectedErrno(err),
        }
    }

    while (true) {
        const rc = system.read(fd, buf.ptr, buf.len);
        switch (errno(rc)) {
            0 => return @intCast(usize, rc),
            EINTR => continue,
            EINVAL => unreachable,
            EFAULT => unreachable,
            EAGAIN => if (std.event.Loop.instance) |loop| {
                loop.waitUntilFdReadable(fd);
                continue;
            } else {
                return error.WouldBlock;
            },
            EBADF => unreachable, // Always a race condition.
            EIO => return error.InputOutput,
            EISDIR => return error.IsDir,
            ENOBUFS => return error.SystemResources,
            ENOMEM => return error.SystemResources,
            ECONNRESET => return error.ConnectionResetByPeer,
            else => |err| return unexpectedErrno(err),
        }
    }
    return index;
}

system in this refers to:

/// When linking libc, this is the C API. Otherwise, it is the OS-specific system interface.
pub const system = if (builtin.link_libc) std.c else switch (builtin.os) {
    .macosx, .ios, .watchos, .tvos => darwin,
    .freebsd => freebsd,
    .linux => linux,
    .netbsd => netbsd,
    .dragonfly => dragonfly,
    .wasi => wasi,
    .windows => windows,
    .zen => zen,
    else => struct {},
};

Here we would change else => struct{}, to look for @import("root").os if it was provided. With this modification, as long as the OS package defined all the constants (such as fd_t and EISDIR), then std.os.write would end up calling the write function from the hobby OS package, and everything Just Works.

Some abstractions may not work so smoothly; in this case there could be code that looks something like this:

fn doTheOsThing() void {
    if (std.builtin.os == .other and @hasDecl(root, "os") and @hasDecl(root.os, "doTheOsThing")) {
        return @import("root").os.doTheOsThing();
    }
}

Really, the check for the "other" OS isn't even necessary. Allowing applications to override fundamental OS functions could be useful on any operating system.

With this proposal implemented, Zig would have very near first class support for all operating systems. The main difference between upstream-recognized OSes and "other" OSes would be where the support is maintained - upstream zig std lib, or in a third party "OS layer" package.

cc @pixelherodev

[jzck]

Great idea for hobby OSes,
Could the idea be pushed even further? All operating system integration could be done via third party packages. This way upstream zig std lib wouldn't even have to know about operating system specifics.

[pixelherodev]

In theory, yes. In practice, that's not feasible - at least, not until the package manager is implemented...

I suppose with the package manager, we could extricate the OS backends, but they'd still need to be upstream, and that just makes things harder to manage.