Use linux/* headers for linux and android

[kamalmarhubi]

This comes out of discussion on #233. Additions to Linux system APIs are included in headers like <linux/header.h> rather than <header.h>. We should use these headers for verification on Linux, in order to include constants that are not present in <header.h>.

@posborne provided some useful background:

Starting with kernel 3.7, there has been a push to separate the headers that are suitable for inclusion from userspace (or at least from libc) into the include/uapi and arch/$ARCH/include/uapi directories in the kernel. This is the case for the constants in this PR: http://lxr.free-electrons.com/source/include/uapi/linux/fcntl.h.

The idea has been around for awhile, but it recently got a bigger push.

As a specific example from #233:

$ printf '#define _GNU_SOURCE\n#include <fcntl.h>' | cpp -dM - | grep '#define F_
GET_SEAL'
$ printf '#define _GNU_SOURCE\n#include <linux/fcntl.h>' | cpp -dM - | grep '#def
ine F_GET_SEAL'
#define F_GET_SEALS (F_LINUX_SPECIFIC_BASE + 10)

[kamalmarhubi]

From reading a bit more, it appears that these uapi headers end up being installed under /usr/include/linux/. However, including that file results in conflicts with the original fcntl.h, which is included via several other headers. To get this to work, we'd have to make a bunch of changes to what headers get used in verification on Linux.
―@kamalmarhubi on #233

[posborne]

However, including that file results in conflicts with the original fcntl.h, which is included via several other headers

You Betcha (I'm feeling very Minnesotan today). This issue will be commonplace when using these headers. My initial thought on this is to have a separate test function with its own ctest::TestGenerator instance that includes the Linux UAPI headers and not ones from $libc. The problem here is that we end up with two sets of tests that don't play particularly nicely together.

If ctest were modified to support an inclusive approach (in addition to the exclusive model that exists currently), I think it might be sane to do the following:

• Define the set of constants, structures, etc. that are to be verified using the Linux headers (possibly with some kind of pattern matching for constants to reduce duplication)
• Exclude these from the linux/musl verification against libc compatability TestGenerator
• Include these for the Linux compatibility TestGenerator

Since the complement of the ignored set from the $libc tests is covered by the linux test, we can have a level of confidence that we have good coverage.

[kamalmarhubi]

@posborne do you know of any disagreements between the libc headers and the UAPI headers on values or signatures they both include? If not, we could just switch to using UAPI headers for verification on (non-musl) Linux, and the other headers on other platforms.

I may be taking a particular reading of the comment, but on #232 @alexcrichton said

The intention of the libc crate was basically to just provide the most up-to-date version of APIs that there are with the assumption that the old APIs are always binary compatible with the new ones (which they typically are). In terms of any fallback implementations, that was going to be left up to downstream crates. Downstream authors need to be vigilant to not use APIs that aren't available on a platform they're shipping on.

[...]

All libc is doing is defining what the APIs would be if they're there.
―@alexcrichton on #232

(Emphasis mine.)

Providing these are part of the kernel's stable ABI, which I understand they are, we should be fine to do this.

[alexcrichton]

Yeah if we need to have two separate test suites here for Linux that's fine by me, also feel free to change whatever needs to be changed in ctest, it was basically created for this project :)

[kamalmarhubi]

This came up in nix-rust/nix#354. @posborne @alexcrichton are there strong objections to just testing against UAPI headers on Linux?

[alexcrichton]

Seems fine to me if they're explicitly intended for inclusion in user-space

[posborne]

No objection from me.

[kamalmarhubi]

@posborne so are the <linux/blah.h> headers definitely for user space use? It seems like yes, since they don't include constants guarded by #ifdef __KERNEL__.

[kamalmarhubi]

This is proving a bit more complicated than I thought. For now I'm unclear on how to use both <fcntl.> (for open(2) and friends), and <linux/fcntl.h> (for file sealing fcntls) in one program.

Eg, this program does not compile.

#define _GNU_SOURCE
#include <fcntl.h>
#include <linux/fcntl.h>

int main(void) {
    int fd = open("/tmp/hi.c", O_RDONLY);
    return 0;
}

[posborne]

@kamalmarhubi Yeah, that is the problem I have run into as well. Although the headers are suitable for use in use space in theory, many of them conflict with libc headers in practice (yuck!). That is why I expressed the thoughts about having a separate test generator in #235 (comment)

The linux exported uapi headers are far from complete, so I do not see an easy way to avoid going down the path of having two separate test suites.