`std::fs::create_dir` not failing on existing directory on aarch64, debug

[yrashk]

I have this piece of code:

src/main.rs:

use std::fs::create_dir;

fn main() {
   create_dir("hello").expect("1");
   create_dir("hello").expect("2");
}

When I tried to run it on aarch64, debug mode, I get empty output. However, I would expect the second except to fail (provided hello didn't exist before).

In fact, when I run the same code with cargo run --release, this is the behavior I get:

$ cargo run --release
    Finished release [optimized] target(s) in 0.04s
     Running `target/release/dir-exists-aarch64`
thread 'main' panicked at '2: Os { code: 17, kind: AlreadyExists, message: "File exists" }', libcore/result.rs:945:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

I've tried it on both stable and nightly:

rustc 1.25.0 (84203cac6 2018-03-25)
binary: rustc
commit-hash: 84203cac67e65ca8640b8392348411098c856985
commit-date: 2018-03-25
host: aarch64-unknown-linux-gnu
release: 1.25.0
LLVM version: 6.0

rustc 1.27.0-nightly (428ea5f6b 2018-05-06)
binary: rustc
commit-hash: 428ea5f6b9e7c6e5ee3294fe9f105e77e89ab407
commit-date: 2018-05-06
host: aarch64-unknown-linux-gnu
release: 1.27.0-nightly
LLVM version: 6.0

Any thoughts?

[cuviper]

Can you run it under strace to see its actual syscalls? Debug and release builds should be identical in this respect, apart from different memory addresses between runs.

[yrashk]

Of course!

Debug:

mkdirat(AT_FDCWD, "hello", 0777)        = 0
mkdirat(AT_FDCWD, "hello", 0777)        = -1 EEXIST (File exists)
newfstatat(AT_FDCWD, "hello", {st_mode=S_IFDIR|0775, st_size=4096, ...}, 0) = 0
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=16384}, NULL) = 0
munmap(0xffffb154a000, 16384)           = 0
exit_group(0)                           = ?
+++ exited with 0 +++

Release:

mkdirat(AT_FDCWD, "hello", 0777)        = 0
mkdirat(AT_FDCWD, "hello", 0777)        = -1 EEXIST (File exists)
write(2, "thread '", 8thread ')                 = 8
write(2, "main", 4main)                     = 4
write(2, "' panicked at '", 15' panicked at ')         = 15
write(2, "2: Os { code: 17, kind: AlreadyE"..., 632: Os { code: 17, kind: AlreadyExists, message: "File exists" }) = 63
write(2, "', ", 3', )                      = 3
write(2, "libcore/result.rs", 17libcore/result.rs)       = 17
write(2, ":", 1:)                        = 1
write(2, "945", 3945)                      = 3
write(2, ":", 1:)                        = 1
write(2, "5", 15)                        = 1
write(2, "\n", 1
)                       = 1
write(2, "note: Run with `RUST_BACKTRACE=1"..., 51note: Run with `RUST_BACKTRACE=1` for a backtrace.
) = 51
futex(0xffffb74cfccc, FUTEX_WAKE_PRIVATE, 2147483647) = 0
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=16384}, NULL) = 0
munmap(0xffffb7534000, 16384)           = 0
exit_group(101)                         = ?
+++ exited with 101 +++

[cuviper]

Well, at least the kernel is behaving as expected. 🙂

I can reproduce this on Fedora 27, using the official Rust 1.25 (w/ LLVM 6). Fedora 27's own rust-1.25.0 package (linked to LLVM 5) does not have the problem! But I also tried Fedora rawhide's package, where it does have LLVM 6, and the problem resurfaces.

[yrashk]

I don't even know where to start on this, to be honest (besides comparing LLVM 5 and 6)... any suggestions?

[yrashk]

And it's interesting that the issue manifests only in debug, not in release

[cuviper]

Something is getting clobbered, or otherwise initialized improperly. Stepping through this in GDB, I found:

std::fs::DirBuilder::create (self=0xffffffffed38, path=0xaaaaaaac7c74 <str.0>) at /checkout/src/libstd/fs.rs:1976
(gdb) p *self
$11 = std::fs::DirBuilder {inner: std::sys::unix::fs::DirBuilder {mode: 511}, recursive: 254}

That mode: 511 is octal 0777 as expected, but recursive is a bool -- it should only be true (1) or false (0), and in this case we expect false! But the generated code behaves as if that 254 is true, then proceeds in the manner of create_dir_all, which doesn't error for existing paths.

[yrashk]

Interesting! But stepping through the same code in release, recursive is 0?

[cuviper]

Under optimization, even with debuginfo the locals are too far gone to inspect. But I can at least see in this bit of code that it's taking the latter path.

   │1979        fn _create(&self, path: &Path) -> io::Result<()> {
   │1980            if self.recursive {
   │1981                self.create_dir_all(path)
   │1982            } else {
  >│1983                self.inner.mkdir(path)
   │1984            }

[yrashk]

Gotcha, as expected. So the main question now is why does recursive become 254. I am pretty sure I don't have any knowledge of internals to suggest anything valuable at this moment. But if you can at least point at the direction you think is worth exploring, I'll try to do my best to help with the research.

[yrashk]

I believe #48673 is somewhat related. I can see 254 there as well!

[yrashk]

Trying out the suggestion from the above issue (wrt to GlobalISel):

$ RUSTFLAGS="-C llvm-args=-fast-isel" cargo run
   Compiling dir-exists-aarch64 v0.1.0 (file:///home/yrashk/dir-exists-aarch64)
    Finished dev [unoptimized + debuginfo] target(s) in 2.73 secs
     Running `target/debug/dir-exists-aarch64`
thread 'main' panicked at '1: Os { code: 17, kind: AlreadyExists, message: "File exists" }', libcore/result.rs:945:5
note: Run with `RUST_BACKTRACE=1` for a backtrace

works as expected!

[cuviper]

OK, that gives something to compare more directly. Both create the DirBuilder the same:

 <_ZN3std2fs10DirBuilder3new17h4d87461ec68baf84E>:
320023e0        orr     w0, wzr, #0x1ff           
2a1f03e1        mov     w1, wzr                   
d65f03c0        ret                               

When the fast-isel create_dir gets that, it just copies w0 and w1 onto the stack:

 <_ZN3std2fs10create_dir17h0c9a5398928d3c8dE>:                                
d10183ff        sub     sp, sp, #0x60                                         
a9057bfd        stp     x29, x30, [sp, #80]                                   
910143fd        add     x29, sp, #0x50                                        
f90017e0        str     x0, [sp, #40]                                         
f9001be1        str     x1, [sp, #48]                                         
f90013e8        str     x8, [sp, #32]                                         
94000f77        bl      7764 <_ZN3std2fs10DirBuilder3new17h4d87461ec68baf84E> 
b9001fe1        str     w1, [sp, #28]                                         
b9001be0        str     w0, [sp, #24]                                         

Under the default (GlobalISel), it sets the precise bits in x8. The bfxil ... #0, #32 sets mode at lsb 0 width 32, and bfi ... #32, #1 sets recursive at lsb 32, width 1. The remaining bits of x8 are untouched! (i.e. garbage) Then x8 is written on the stack in the same place as before.

 <_ZN3std2fs10create_dir17h0c9a5398928d3c8dE>:                               
d10183ff        sub     sp, sp, #0x60                                        
a9057bfd        stp     x29, x30, [sp, #80]                                  
910143fd        add     x29, sp, #0x50                                       
9100a3e9        add     x9, sp, #0x28                                        
f9000120        str     x0, [x9]                                             
f9001be1        str     x1, [sp, #48]                                        
f90013e8        str     x8, [sp, #32]                                        
94000fcc        bl      79f4 <_ZN3std2fs10DirBuilder3new17h4d87461ec68baf84E>
2a0003e9        mov     w9, w0                                               
b3407d28        bfxil   x8, x9, #0, #32                                      
2a0103e9        mov     w9, w1                                               
b3600128        bfi     x8, x9, #32, #1                                      
f9000fe8        str     x8, [sp, #24]                                        

Then DirBuilder::_create looks the same for both: a condition on the entire byte at offset #4 branching into the different tail calls. But with the GlobalISel, this is where we see our 254, 0b11111110 where only the lsb 0 was set for our false.

 <_ZN3std2fs10DirBuilder7_create17h1220ff0bdc1b9d5bE>:                                   
39401009        ldrb    w9, [x0, #4]                                                     
34000049        cbz     w9, 7a0c <_ZN3std2fs10DirBuilder7_create17h1220ff0bdc1b9d5bE+0xc>
14000002        b       7a10 <_ZN3std2fs10DirBuilder14create_dir_all17hdbee7d289e9930e9E>
14001ef2        b       f5d4 <_ZN3std3sys4unix2fs10DirBuilder5mkdir17ha85b362192bd0679E> 

[cuviper]

Even this shows the wrong recursive:

fn main() {
    println!("{:?}", std::fs::DirBuilder::new());
}

DirBuilder { inner: DirBuilder { mode: 511 }, recursive: true }