internal compiler error: cannot lex `source_file` without source: /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/ppv-lite86-0.2.6/src/x86_64/mod.rs

[staktrace]

The setup under which this occurs is somewhat complicated, but I'll try to reduce it as much as I can and update this issue with details. But basically it's using the latest nightly rustc and running cargo check on the latest glean code (https://github.com/mozilla/glean, rev a30e58cebe6888a6c15fcb58d32dc00a9c38bc17).

Code

RUSTFLAGS='-Z save-analysis' cargo +nightly check --all --target-dir=/mnt/index-scratch/glean/objdir

Meta

rustc --version --verbose:

rustc 1.43.0-nightly (158127853 2020-03-10)
binary: rustc
commit-hash: 15812785344d913d779d9738fe3cca8de56f71d5
commit-date: 2020-03-10
host: x86_64-unknown-linux-gnu
release: 1.43.0-nightly
LLVM version: 9.0

Error output

error: internal compiler error: cannot lex `source_file` without source: /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/ppv-lite86-0.2.6/src/x86_64/mod.rs

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.43.0-nightly (158127853 2020-03-10) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z save-analysis -C debuginfo=2 --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
error: aborting due to previous error

error: could not compile `c2-chacha`.

(See full output in the backtrace details below)

Backtrace

 Downloading crates ...
  Downloaded bincode v1.2.1
  Downloaded regex v1.3.4
  Downloaded chrono v0.4.10
  Downloaded uuid v0.8.1
  Downloaded serde v1.0.104
  Downloaded once_cell v1.3.1
  Downloaded env_logger v0.7.1
  Downloaded serde_json v1.0.48
  Downloaded rkv v0.10.3
  Downloaded ffi-support v0.4.0
  Downloaded num-integer v0.1.42
  Downloaded regex-syntax v0.6.16
  Downloaded libc v0.2.67
  Downloaded serde_derive v1.0.104
  Downloaded byteorder v1.3.4
  Downloaded arrayref v0.3.6
  Downloaded ryu v1.0.2
  Downloaded termcolor v1.1.0
  Downloaded quick-error v1.2.3
  Downloaded atty v0.2.14
  Downloaded itoa v0.4.5
  Downloaded rand v0.7.3
  Downloaded failure v0.1.6
  Downloaded ordered-float v1.0.2
  Downloaded rand_core v0.5.1
  Downloaded failure_derive v0.1.6
  Downloaded proc-macro2 v1.0.9
  Downloaded bitflags v1.2.1
  Downloaded url v2.1.1
  Downloaded syn v1.0.16
  Downloaded lmdb-rkv v0.12.3
  Downloaded rand_chacha v0.2.1
  Downloaded getrandom v0.1.14
  Downloaded idna v0.2.0
  Downloaded c2-chacha v0.2.3
  Downloaded percent-encoding v2.1.0
  Downloaded lmdb-rkv-sys v0.9.6
  Downloaded unicode-normalization v0.1.12
  Downloaded ppv-lite86 v0.2.6
  Downloaded cc v1.0.50
  Downloaded pkg-config v0.3.17
  Downloaded smallvec v1.2.0
   Compiling libc v0.2.67
   Compiling proc-macro2 v1.0.9
   Compiling unicode-xid v0.2.0
   Compiling syn v1.0.16
    Checking cfg-if v0.1.10
   Compiling autocfg v1.0.0
   Compiling getrandom v0.1.14
   Compiling serde v1.0.104
   Compiling byteorder v1.3.4
   Compiling pkg-config v0.3.17
    Checking ppv-lite86 v0.2.6
   Compiling cc v1.0.50
   Compiling bitflags v1.2.1
   Compiling log v0.4.8
    Checking smallvec v1.2.0
   Compiling failure_derive v0.1.6
    Checking matches v0.1.8
   Compiling ryu v1.0.2
    Checking percent-encoding v2.1.0
    Checking lazy_static v1.4.0
    Checking itoa v0.4.5
    Checking regex-syntax v0.6.16
    Checking arrayref v0.3.6
    Checking once_cell v1.3.1
    Checking quick-error v1.2.3
    Checking termcolor v1.1.0
    Checking unicode-normalization v0.1.12
    Checking unicode-bidi v0.3.4
   Compiling num-traits v0.2.11
   Compiling num-integer v0.1.42
    Checking humantime v1.3.0
    Checking c2-chacha v0.2.3
error: internal compiler error: cannot lex `source_file` without source: /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/ppv-lite86-0.2.6/src/x86_64/mod.rs

thread 'rustc' panicked at 'Box<Any>', src/librustc_errors/lib.rs:880:9
stack backtrace:
    Checking ffi-support v0.4.0
   Compiling quote v1.0.2
    Checking idna v0.2.0
   0:     0x7fd8b7d5bea4 - backtrace::backtrace::libunwind::trace::h79b6ad55bf71ea4d
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1:     0x7fd8b7d5bea4 - backtrace::backtrace::trace_unsynchronized::hcfa816a17c383c01
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2:     0x7fd8b7d5bea4 - std::sys_common::backtrace::_print_fmt::h1fbf4e8dad31c04d
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x7fd8b7d5bea4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h691b7317de5d1611
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x7fd8b7d9489c - core::fmt::write::h0edcddf75fa815b5
                               at src/libcore/fmt/mod.rs:1063
   5:     0x7fd8b7d4dbc3 - std::io::Write::write_fmt::hbcb4a18a26d43db2
                               at src/libstd/io/mod.rs:1426
   6:     0x7fd8b7d60cd5 - std::sys_common::backtrace::_print::h6ea84893fd9c4f5c
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x7fd8b7d60cd5 - std::sys_common::backtrace::print::hb27542ff92a9665f
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x7fd8b7d60cd5 - std::panicking::default_hook::{{closure}}::hdda292fa66c32722
                               at src/libstd/panicking.rs:204
   9:     0x7fd8b7d60a22 - std::panicking::default_hook::hb282c50c9862e02d
                               at src/libstd/panicking.rs:224
  10:     0x7fd8b83b90c9 - rustc_driver::report_ice::h556269f1013a04f1
  11:     0x7fd8b7d61405 - std::panicking::rust_panic_with_hook::hbe9819d9d894172d
                               at src/libstd/panicking.rs:474
  12:     0x7fd8bab017b3 - std::panicking::begin_panic::h39b92cee9cd52b71
  13:     0x7fd8baaca750 - rustc_errors::HandlerInner::bug::ha8fcca766396bc52
  14:     0x7fd8baac94c0 - rustc_errors::Handler::bug::h524f2cc66bc71cfd
  15:     0x7fd8b9dc7989 - rustc_parse::lexer::StringReader::new::h83554f3f7ce8325c
  16:     0x7fd8b9dc7b97 - rustc_parse::lexer::StringReader::retokenize::haf8932c92ef7f0cb
  17:     0x7fd8b84f789c - rustc_save_analysis::span_utils::SpanUtils::sub_span_of_token::h3680c12557eb839e
  18:     0x7fd8b84b5544 - rustc_save_analysis::dump_visitor::DumpVisitor::process_use_tree::h17df36bb5159a5b2
  19:     0x7fd8b84b731d - <rustc_save_analysis::dump_visitor::DumpVisitor as rustc_ast::visit::Visitor>::visit_item::h415b0b015651a8cb
  20:     0x7fd8b84bbf0f - <rustc_save_analysis::dump_visitor::DumpVisitor as rustc_ast::visit::Visitor>::visit_item::h415b0b015651a8cb
  21:     0x7fd8b84b900a - <rustc_save_analysis::dump_visitor::DumpVisitor as rustc_ast::visit::Visitor>::visit_item::h415b0b015651a8cb
  22:     0x7fd8b84b6a2a - <rustc_save_analysis::dump_visitor::DumpVisitor as rustc_ast::visit::Visitor>::visit_mod::hb3951d0b06d3a669
  23:     0x7fd8b8440ff3 - rustc_ast::visit::walk_crate::h7559d33fc73f1f47
  24:     0x7fd8b842d05e - rustc::dep_graph::graph::DepGraph::with_ignore::h32d5b32739cb0104
  25:     0x7fd8b840102e - rustc_session::utils::<impl rustc_session::session::Session>::time::hdf63c34b18b22e5e
  26:     0x7fd8b843df0b - rustc::ty::context::tls::enter_global::h992246c787b278e1
  27:     0x7fd8b840e263 - rustc_interface::interface::run_compiler_in_existing_thread_pool::h84d51f9f8b68c8ca
  28:     0x7fd8b83c25bd - scoped_tls::ScopedKey<T>::set::h4cb060ee70710273
  29:     0x7fd8b83bf514 - rustc_ast::attr::with_globals::h10cf6ded6d7b02ee
  30:     0x7fd8b83c3c00 - std::sys_common::backtrace::__rust_begin_short_backtrace::h72f84d91a30b8442
  31:     0x7fd8b7d726d7 - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:86
  32:     0x7fd8b84110d6 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h17127d11297e0161
  33:     0x7fd8b7d3d66f - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::hce6b09a53f302609
                               at /rustc/15812785344d913d779d9738fe3cca8de56f71d5/src/liballoc/boxed.rs:1017
  34:     0x7fd8b7d70fdd - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h7d11f09ec1b28d83
                               at /rustc/15812785344d913d779d9738fe3cca8de56f71d5/src/liballoc/boxed.rs:1017
  35:     0x7fd8b7d70fdd - std::sys_common::thread::start_thread::h5a725994801096bc
                               at src/libstd/sys_common/thread.rs:13
  36:     0x7fd8b7d70fdd - std::sys::unix::thread::Thread::new::thread_start::h84c763d83a5c132e
                               at src/libstd/sys/unix/thread.rs:80
  37:     0x7fd8b7ad76db - start_thread
  38:     0x7fd8b73f488f - __clone
  39:                0x0 - <unknown>

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.43.0-nightly (158127853 2020-03-10) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z save-analysis -C debuginfo=2 --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
error: aborting due to previous error

error: could not compile `c2-chacha`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

[staktrace]

Also to note: so far this has happened 2/2 times today. It didn't happen yesterday (which presumably was running using a slightly-older rustc and checking slightly-older glean code).

[staktrace]

Re-running the command:

RUSTFLAGS="-Z save-analysis" cargo +nightly check --all --target-dir=/mnt/index-scratch/glean/objdir

in the proper directory on the machine in question reproduces the error. The file it's complaining about (/home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/ppv-lite86-0.2.6/src/x86_64/mod.rs) exists and doesn't appear corrupted.

[staktrace]

Running the command over and over changes which crate it errors out on. In the initial run it was erroring out on c2-chacha but on subsequent runs it changes to synstructure, serde_derive, etc.

[staktrace]

If I remove the RUSTFLAGS="-Z save-analysis" bit then it works fine. So seems related to the save-analysis code.

/cc @Xanewok

[staktrace]

I'm able to reproduce this in a local environment as well. Minimal steps so far (on a Ubuntu 18.04 VM)

rustup update nightly-2020-03-11 # actually installs a rustc where --version reports 2020-03-10
rustup default nightly-2020-03-11
git clone https://github.com/mozilla/glean
cd glean
git reset --hard a30e58cebe6888a6c15fcb58d32dc00a9c38bc17
RUSTFLAGS="-Z save-analysis" cargo check

I ran cargo with --verbose and the specific rustc invocation that's reliably reproducing the failure is:

rustc --crate-name c2_chacha --edition=2018 /home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/c2-chacha-0.2.3/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata -C debuginfo=2 --cfg 'feature="simd"' --cfg 'feature="std"' -C metadata=8cbde2cd3443e18e -C extra-filename=-8cbde2cd3443e18e --out-dir /home/vagrant/mozilla-index/glean/glean/target/debug/deps -L dependency=/home/vagrant/mozilla-index/glean/glean/target/debug/deps --extern ppv_lite86=/home/vagrant/mozilla-index/glean/glean/target/debug/deps/libppv_lite86-ff6fe2823b6b9168.rmeta --cap-lints allow -Z save-analysis

(Obviously your paths may vary)

Also note that this does NOT happen with rustup default nightly-2020-03-10 (which installs a rustc built on 2020-03-09) so it's a regression between rustc version 3dbade652 and rustc version 158127853.

[staktrace]

Rust regression range: 3dbade6...1581278

[staktrace]

If you're unable to repro or need more detail feel free to let me know. Otherwise for now I'm leaving this with you (i.e. rust developers) to dig into.

[sisshiki1969]

I have the very same crush with rustc 1.43.0-nightly (c20d7ee 2020-03-11) on both of Mac and WSL2/Ubuntu, every time Rust language Server starts up in VSCode.

$rustc --version --verbose
rustc 1.43.0-nightly (c20d7eecb 2020-03-11)
binary: rustc
commit-hash: c20d7eecbc0928b57da8fe30b2ef8528e2bdd5be
commit-date: 2020-03-11
host: x86_64-unknown-linux-gnu
release: 1.43.0-nightly
LLVM version: 9.0

• No problem in rustc 1.43.0-nightly (beac68a 2020-03-01).

$rls --version
rls 1.41.0 (5fde462 2020-02-21)

Error message:

{"message":"cannot lex `source_file` without source: /home/sisshiki/.cargo/registry/src/github.com-1ecc6299db9ec823/ppv-lite86-0.2.6/src/x86_64/mod.rs","code":null,"level":"error: internal compiler error","spans":[],"children":[],"rendered":"\u001b[0m\u001b[1m\u001b[38;5;9merror: internal compiler error\u001b[0m\u001b[0m\u001b[1m: cannot lex `source_file` without source: /home/sisshiki/.cargo/registry/src/github.com-1ecc6299db9ec823/ppv-lite86-0.2.6/src/x86_64/mod.rs\u001b[0m\n\n"}
thread 'rustc' panicked at 'Box<Any>', src/librustc_errors/lib.rs:880:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.43.0-nightly (c20d7eecb 2020-03-11) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 --crate-type lib

note: some of the compiler flags provided by cargo are hidden

{"message":"aborting due to previous error","code":null,"level":"error","spans":[],"children":[],"rendered":"\u001b[0m\u001b[1m\u001b[38;5;9merror\u001b[0m\u001b[0m\u001b[1m: aborting due to previous error\u001b[0m\n\n"}

[staktrace]

I used cargo bisect-rustc to narrow the range to dd155df...1581278