Skip to content

Add statx shim for linux target #1101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 22, 2019
Merged

Add statx shim for linux target #1101

merged 1 commit into from
Dec 22, 2019

Conversation

pvdrz
Copy link
Contributor

@pvdrz pvdrz commented Dec 4, 2019

This is an attempt to fix: #999 (for linux only)

Currently there is one problem that I haven't been able to solve. std::fs::metadata fails because the creation time is not available even though it is provided in the shim code.

In order to inform the caller that the field was provided, the stx_flag field must have the bits of STATX_BTIME set (which they are). The creation time is in the stx_btime field of the statx struct (see [1]). The relevant code in libstd is here (probably?): https://github.com/rust-lang/rust/blob/master/src/libstd/sys/unix/fs.rs#L322

Another important point is that we are just providing the fields that are available in "all" platforms (this is, without using any platform specific traits or so). This can be improved later.

References:
[1] Man page: http://man7.org/linux/man-pages/man2/statx.2.html
[2] libc statx struct: https://docs.rs/libc/0.2.63/libc/struct.statx.html

Edit: The problem is that my filesystem is not providing it and I thought all filesystems could provide it. I changed the code so it only provides those dates if they are available. now we are ready to go.

r? @RalfJung @oli-obk

@pvdrz pvdrz changed the title [WIP] Add statx shim for linux target Add statx shim for linux target Dec 4, 2019
@pvdrz
Copy link
Contributor Author

pvdrz commented Dec 5, 2019

Hmm apparently OSX uses stat64. I'm going to do some intermediate functions to implement it. I believe I can test it by disabling statx.

Edit: I implemented stat64 for linux and of course OSX's is different. I tried to follow apple's manual the best I could. I hope it works.

Edit2: Apparently the specifics of the stat64 struct in OSX is not specified well enough in the manual. I wasn't able to put the fields in the right order. I'm going to remove the stat64 shim and exclude OSX from the metadata test for now. Someone who has OSX might be able to debug this without wasting CI time D:

@pvdrz pvdrz force-pushed the stat-shim branch 7 times, most recently from cd958e5 to 61272f0 Compare December 5, 2019 15:40
@RalfJung RalfJung mentioned this pull request Dec 22, 2019
src/shims/fs.rs Outdated
Comment on lines 307 to 309
// This argument should be a `c_int` but the syscall API just returns a pointer sized
// integer.
let flags: i32 = this.try_from_ptr_sized_operand(flags_op)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is that statx call site? I seems just wrong to me to pass an isize when the syscall says it should be a c_int.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the reason these are all isize is that they all get cast right here. That is very odd.

@gnzlbg any idea why libstd turns all syscall parameters to c_long, even when the official syscall docs use other types such as "int" or "unsinged"?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ehm, no idea, the first argument to glibc's syscall is a c_long specifying the number of arguments, but the rest of the arguments are a C variadic list of arguments, and they should have the correct types of the syscall. If the syscall expects an c_int, and you pass it an isize, then that would have the wrong argument size on, e.g., 64-bit linux, so that seems wrong. Maybe it doesn't "matter" in practice, e.g., due to how the calling convention handles these arguments, but still, looks like a bug. cc @Amanieu

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first argument seems to be the syscall number, not the number of arguments?

If the syscall expects an c_int, and you pass it an isize, then that would have the wrong argument size on, e.g., 64-bit linux, so that seems wrong.

I think that is exactly what is happening. All parameters of all syscalls going through that macro are passed as c_long, no matter their actual type.

src/shims/fs.rs Outdated
// The `mode` field specifies the type of the file and the permissions over the file for
// the owner, its group and other users. Given that we can only provide the file type
// without using platform specific methods, we only set the bits corresponding to the file
// type. This argument should be an `__u16` but the syscall API just returns a pointer sized
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"the syscall API" doesn't return anything here, does it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that's just bad wording from my part. "provides" maybe?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thus is not about "the syscall API" at all. The thing is that syscall is an untyped function so all types are determined by the caller, and the particular caller we are working with here decided to make everything an isize. (I am still trying to figure out why.)
This is not a universal truth for all syscalls.

@RalfJung
Copy link
Member

I think I grilled you long enough, let's go ahead and land this. :)

@bors r+

@bors
Copy link
Contributor

bors commented Dec 22, 2019

📌 Commit b0c7625 has been approved by RalfJung

@bors
Copy link
Contributor

bors commented Dec 22, 2019

⌛ Testing commit b0c7625 with merge a3ea1cb...

bors added a commit that referenced this pull request Dec 22, 2019
Add statx shim for linux target

This is an attempt to fix: #999 (for linux only)

Currently there is one problem that I haven't been able to solve. `std::fs::metadata` fails because the creation time is not available even though it is provided in the shim code.

In order to inform the caller that the field was provided, the `stx_flag` field must have the bits of `STATX_BTIME` set (which they are). The creation time is in the `stx_btime` field of the `statx` struct (see [1]). The relevant code in `libstd` is here (probably?): https://github.com/rust-lang/rust/blob/master/src/libstd/sys/unix/fs.rs#L322

Another important point is that we are just providing the fields that are available in "all" platforms (this is, without using any platform specific traits or so). This can be improved later.

References:
[1] Man page: http://man7.org/linux/man-pages/man2/statx.2.html
[2] libc `statx` struct: https://docs.rs/libc/0.2.63/libc/struct.statx.html

Edit: The problem is that my filesystem is not providing it and I thought all filesystems could provide it. I changed the code so it only provides those dates if they are available. now we are ready to go.

r? @RalfJung @oli-obk
@bors
Copy link
Contributor

bors commented Dec 22, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: RalfJung
Pushing a3ea1cb to master...

@bors bors merged commit b0c7625 into rust-lang:master Dec 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

macOS: Implement stat64 to get rid of the termcap hack
5 participants