ICE: assertion failed in ppc64 make_indirect

[cuviper]

Using:

$ rustc +nightly -Vv
rustc 1.19.0-nightly (10d7cb44c 2017-06-18)
binary: rustc
commit-hash: 10d7cb44c98f25c04dcefb6b6555237de8b8bd7e
commit-date: 2017-06-18
host: powerpc64-unknown-linux-gnu
release: 1.19.0-nightly
LLVM version: 4.0

There's an assertion failed compiling run-make/extern-fn-struct-passing-abi:

$ RUST_BACKTRACE=1 rustc +nightly src/test/run-make/extern-fn-struct-passing-abi/test.rs
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.19.0-nightly (10d7cb44c 2017-06-18) running on powerpc64-unknown-linux-gnu

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'assertion failed: `(left == right)` (left: `Indirect`, right: `Direct`)', /checkout/src/librustc_trans/abi.rs:444
stack backtrace:
   0: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libstd/sys_common/backtrace.rs:71
   2: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:355
   3: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libstd/panicking.rs:365
   4: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libstd/panicking.rs:549
   5: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libstd/panicking.rs:511
   6: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libstd/panicking.rs:495
   7: rust_metadata_rustc_trans_688c63823f70e585d77215bf3397b7f5
   8: rust_metadata_rustc_trans_688c63823f70e585d77215bf3397b7f5
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: rust_metadata_rustc_trans_688c63823f70e585d77215bf3397b7f5
  13: <unknown>
  14: <unknown>
  15: <unknown>
  16: <unknown>
  17: <unknown>
  18: <unknown>
  19: <unknown>
  20: <unknown>
  21: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/libpanic_unwind/lib.rs:98
  22: <unknown>
  23: rust_metadata_std_370a936915a1568ed47373dc73e14e2d
             at /checkout/src/liballoc/boxed.rs:660
             at /checkout/src/libstd/sys_common/thread.rs:21
             at /checkout/src/libstd/sys/unix/thread.rs:84
  24: <unknown>
  25: <unknown>

(Apparently backtracing isn't so hot either...)

The failed assertion is here in librustc_trans/abi.rs, make_indirect().

I only have this problem on ppc64 (big-endian), not ppc64le, so I suspect it may be coming from librustc_trans/cabi_powerpc64.rs classify_ret_ty(). The first call to make_indirect() is only for big-endian, and then the second call seems like it will trigger the assertion for being called on something already indirect.

I can reproduce this with native ppc64 back to 1.14.0, which is the first ppc64 host rustc. I haven't tried cross-compiling to ppc64, but I suspect that will trigger the issue too.

[cuviper]

It does ICE when cross-compiling too:

$ rustc +nightly -Vv
rustc 1.20.0-nightly (f590a44ce 2017-06-27)
binary: rustc
commit-hash: f590a44ce61888c78b9044817d8b798db5cd2ffd
commit-date: 2017-06-27
host: x86_64-unknown-linux-gnu
release: 1.20.0-nightly
LLVM version: 4.0

$ RUST_BACKTRACE=1 rustc +nightly --target powerpc64-unknown-linux-gnu src/test/run-make/extern-fn-struct-passing-abi/test.rs
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.20.0-nightly (f590a44ce 2017-06-27) running on x86_64-unknown-linux-gnu

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'assertion failed: `(left == right)`
  left: `Indirect`,
 right: `Direct`', /checkout/src/librustc_trans/abi.rs:444
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at /checkout/src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:355
   3: std::panicking::default_hook
             at /checkout/src/libstd/panicking.rs:365
   4: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:549
   5: std::panicking::begin_panic
             at /checkout/src/libstd/panicking.rs:511
   6: std::panicking::begin_panic_fmt
             at /checkout/src/libstd/panicking.rs:495
   7: rustc_trans::abi::ArgType::make_indirect
   8: rustc_trans::abi::FnType::adjust_for_abi
   9: rustc_trans::mir::block::<impl rustc_trans::mir::MirContext<'a, 'tcx>>::trans_terminator
  10: rustc_trans::mir::trans_mir
  11: rustc_trans::trans_item::TransItem::define
  12: rustc_trans::base::trans_crate
  13: rustc_driver::driver::phase_4_translate_to_llvm
  14: rustc_driver::driver::compile_input::{{closure}}
  15: rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}
  16: rustc_driver::driver::phase_3_run_analysis_passes
  17: rustc_driver::driver::compile_input
  18: rustc_driver::run_compiler

[cuviper]

This is now affecting upstream Firefox when compiling clang-sys:
https://bugzilla.redhat.com/show_bug.cgi?id=1482447

[cuviper]

I'm working on this -- besides this direct/indirect error, there are several more adjustments needed for differences between PowerPC ELF v1 and v2 ABIs (big-endian and little-endian).