Generating libc from C headers

[Zoxc]

It would be nice if there was a way to generate libc using just a standard C library. That would make porting to less common targets way easier and also ensures that the headers are correct.

[alexcrichton]

I don't think there's necessarily a large amount of motivation for this in the sense that this entire library is already verified as correct and header -> Rust generators I've seen tend to have lots of things that need to be fixed by hand. It's certainly plausible to generate headers and then commit them, but I don't think there's any particular reason to rewrite the crate to use generation now.

[Zoxc]

It doesn't support newlib currently (or more obscure C libraries), which makes porting libstd more difficult.

[kristate]

Trying to clean-up issues: is this still an issue @Zoxc ?

[Zoxc]

@kristate It is.

[gnzlbg]

Platforms break ABI compatibility all the time due to changes in the values of constants, enums, sizes and alignment of structs, even signature of functions.

As long as libc does not use a binding generation, every single libc version targets a specific version of an API, and has undefined behavior on all others.

[gnzlbg]

@Zoxc This is not a bad idea. The main problems are that generating libc from headers requires:

• re-compiling standard, so this is only (at least for now) relevant for those using xargo
• using rust-bindgen requires a particular clang toolchain (which, at least for now, rustup is not bundling)

If one needs to re-compile libstd anyways, one can [patch]/[source] it to use a different "libc", e.g., "libc-bindgen", which uses rust-bindgen to generate Rust FFI from headers.

I think it would be better to create such a "libc-bindgen" first, add minimal support to xargo to patch libstd (and all other crates) to use it in place of libc, and then explore solutions for the problems that will arise (like improving bindgen), e.g., due to differences in API between libc and the result of bindgen.

Once we are there, we could start considering whether it is worth integrating such a thing with libc somehow. But I think that right now it would be too soon to discuss such a thing because we lack information about the problems / issues that will need to be solved for that to happen.

Another approach that such a different libc-bindgen library could support is using that to generate Rust headers for an environment like newlib, and then statically import those into the libc repo, as a way of quickly gaining comprehensive support for new platforms.

As @alexcrichton mentions, I doubt that this is a process that can be fully automated. Bindgen has made huge leaps in the last couple of years, but even for tiny well-behaved C libraries I've needed to perform significant manual modifications when attempting to do similar things. Platform headers are anything but well-behaved and use a lot of platform specific C compiler features, that might cause clang and/or bindgen significant trouble.

[gnzlbg]

I'd be glad to help anybody wanting to pursue such an endeavor, but I think that it might be best to close this issue for now, because until somebody puts in the effort to come up with a proof-of-concept of this, we just know too little about which problems need to be solved for us to have a fruitful discussion about this.