Consider adding Linux targets that don't depend on libc

[japaric]

e.g. x86_64-linux (or x86_64-unknown-linux-rust).

These would be the spiritual successor of steed: a standard library, free of C dependencies, for Linux systems. Steed implemented (or planned to) the Rust standard library using raw Linux system calls instead of libc's API. These libc-less targets would also implement std using raw system calls.

Even though steed has been inactive for over a year people continue to star it on GitHub and currently has over 500 stars so it seems there's still interest for something like it.

What we learned during development is that maintaining a standard library out of tree is a lot of work because things get quickly out sync so if there's still interest for something like steed I would recommend writing an RFC to add support for libc-less targets (e.g. x86_64-linux) to rust-lang/rust; this would be equivalent to importing and developing steed as part of rust-lang/rust.

An RFC is also a good way to (re-)evaluate the ROI of making a libc-less standard library. One of the goals of steed was hassle-free cross compilation but today that's solved by the Linux/MUSL targets + rust-lld (works on stable). The other goal was better optimizations (plain LTO doesn't optimize across FFI) but cross-language LTO is a now thing (-Z cross-lang-lto). There may be less costly ways to achieve the same results.

The rest of this post describes the status of steed as of 2017-10-20 (date of last commit).

What we had working:

1. standard I/O (std::io::{stdin,stderr,stdout})
2. filesystem operations (std::fs)
3. collections (std::collections) (but see (a) below)
4. std::sync::Mutex (courtesy of parking_lot)
5. std::env
6. UDP and TCP sockets (std::net) (but see (d) below)
7. minimal support for threads and TLS
8. #[test] support (but see (c) below)

You can check the examples directory to get an idea of what you could write with it.

What was missing:

a. a proper allocator. steed used a bump pointer allocator that never freed memory
b. all the math stuff (e.g. f32::sin). These days one can use libm.
c. unwinding, all the steed targets used -C panic=abort
d. hostname lookup
e. errno. It was unclear whether we should implement it or not. It seems to only be required by std::io::Error::last_os_error. Linux system calls have a Result-like API so errno is not required to propagate errors from syscalls to the std API.

and much more stuff; you can check the issue tracker for the full list.

---

cc @tbu- @briansmith

[mark-i-m]

This sounds really cool! However, I think you may be underestimating how much work this is. There would also need to be collaboration with kernel developers about system calls and stuff.

[lorenz]

@mark-i-m I agree that it is quite a bit of work, but there is no need for collaborating with kernel devs. The syscall interface is considered stable and they treat any change that'll break userspace as a bug. Go has successfully implemented this strategy on all OSs they support. So I'd argue it is pretty doable and in my opinion desirable.

[tbu-]

This sounds really cool! However, I think you may be underestimating how much work this is. There would also need to be collaboration with kernel developers about system calls and stuff.

Most of the stuff was already in place, as you can see in @japaric's comment. :)

[mark-i-m]

@lorenz Yes, you are right, but what I mean is that if we want to influence the syscall interfaces to make them more Rust-friendly we would need to interact more with the kernel devs...

[lorenz]

@mark-i-m What do you have in mind that you want changed? The syscall interface is totally usable from Rust and is in various ways a better fit than libc itself.

[mark-i-m]

Admittedly, I haven't yet gotten to thinking extensively about this, but mainly my thinking is that syscalls often don't even attempt to by safe, so adding good Rust wrappers around them is either painful or inefficient.

For example, it is hard to write a good safe wrapper for mmap because it can do so many things; encoding all of them into efficient zero-cost abstractions is hard. There are also a bunch of syscalls that depend on C-defined structs/types (and their layouts), e.g. clock_get*, sched_setaffinity. (EDIT: actually, I'm not sure if these structs/types are just parts of libc... so I might be wrong there...)

[lorenz]

But this is not about implementing all possible safe uses of (for example) mmap, but just about reimplementing the standard library on top of syscalls which limits the scope to these specific operations. Also structs with C layout are no problem in Rust (we have #[repr(C)]) and are a sensible choice for a kernel since they are interoperable with everything. The kernel people will never maintain a separate syscall interface just for a specific programming language.

[mark-i-m]

The kernel people will never maintain a separate syscall interface just for a specific programming language.

No, but they do talk with e.g. libc maintainers to discuss interfaces, libc support, etc.

But this is not about implementing all possible safe uses of (for example) mmap, but just about reimplementing the standard library on top of syscalls which limits the scope to these specific operations.

That's true, but this is also an opportunity to expose the kernel ABI in a safer way, although that doesn't seem to be the intent of this issue...

[SimonSapin]

Any newly-proposed syscall will at best be present in the next kernel version, but it’ll be a number of years before you can usefully ship a program that just assumes it is present.

[briansmith]

Any newly-proposed syscall will at best be present in the next kernel version, but it’ll be a number of years before you can usefully ship a program that just assumes it is present.

Yes, but Rust can take advantage of them right away by falling back to something else when it receives ENOSYS (or whatever), without having to rely on convincing glibc/musl maintainers to do so.

[SimonSapin]

Sure but if a fallback is needed anyway there is no point in doing all this only "to make them more Rust-friendly", is there? (As opposed to, say, a performance optimization.)

[joshtriplett]

@mark-i-m It sounds like you're interested in something different than this is proposing. @japaric is proposing a standard library that's equivalent to the current std but without linking to the platform libc, by calling existing syscalls directly. That's a useful goal that many people want, and it doesn't involve changes to syscalls in any way.

If someone wanted to add new syscalls to Linux, for whatever reason, that would be outside the scope of this proposal.

[joshtriplett]

As a separate thought, for a target like this, we could theoretically have a libc-compatible C library implemented in Rust, and then link C programs to that library. We don't need that, but it would make for an interesting future addition to this target.

[Darkspirit]

libc-compatible C library implemented in Rust

https://gitlab.redox-os.org/redox-os/relibc#relibc
https://github.com/redox-os/relibc

"relibc is a portable POSIX C standard library written in Rust."

It is under heavy development, and currently supports Redox and Linux.

The motivation for this project is twofold: Reduce issues the redox crew was having with newlib, and create a safer alternative to a C standard library written in C. It is mainly designed to be used under redox, as an alternative to newlib, but it also supports linux syscalls via the sc crate.

Supported OSes
Redox OS
Linux

Supported architectures
x86_64
Aarch64

Edit:
Just found this: This tries to add an x86_64-unknown-linux-relibc target:
mati865/rust#1 (comment)
mati865/rust@master...relibc
rust-lang/libc@master...mati865:relibc

[newpavlov]

Looking forward to playing with this libc-free target! I wonder if it will be possible to later remove x86_64-unknown-linux reliance on libc after experimenting on a separate target?