Figure out why we are printing fully qualified paths for std library for tests

[celinval]

We would like to be able to enable a few test suites from rustc in our repository to provide good test coverage. Unfortunately, the ui based suites (modes ui, mir-opt, and pretty) have a lot of tests failing because of how our test driver prints fully qualified paths for standard library.

E.g.: The MIR output diff look like:

-    _10 = _eprint(move _11) -> [return: bb6, unwind unreachable];
+    _10 = std::io::_eprint(move _11) -> [return: bb6, unwind unreachable];

[b-naber]

The reason that the defpaths aren't trimmed in our test-driver is that we don't use the TimePassesCallbacks, which sets trimmed_def_paths. This option is later used when deciding whether to trim defpaths. Not sure what the right fix is here; we could add the Callback or maybe rustc should have an option to set that (currently this is always set to false and the TimePassesCallbacks is the only way to set this).

Edit: mh there is an unstable option for trim_diagnostic_paths. I feel that that option should be sufficient to trim those paths?!

[celinval]

Nice! We should try that

[b-naber]

I looked into this some more, and there's a problem. Currently trimmed_def_path is set to false by default, and there's not a command-line option for it. The reason for this is that there's an assertion that tries to ensure that the trimmed_def_paths query (which is expensive) is only called for diagnostics. When some analysis queries are 'ensured' they seem to somehow wind up printing a string, and if we were to set the trimmed_def_paths flag these would panic (haven't investigated why they would print strings, but I'm pretty sure that this is the reason for the ICE). As a way around trimming def paths in those print calls, rustc currently sets that flag in TimePassesCallbacks, which would supposedly be run after those queries?! See e.g. rust-lang/rust-clippy#7668 where they ran into this problem too.

So the only way we could fix this would be to add a Callback in stable mir that sets that flag. Do you have any thoughts on what would be the best way to do this? Should we have a separate test_run macro maybe?

EDIT: hm no, actually using the callback doesn't solve this either, since its config is used before the compiler is actually run. Don't know why this doesn't ICE in rustc then. This is kind of a pain to debug, how important is this issue?

[bjorn3]

As a way around trimming def paths in those print calls, rustc currently sets that flag in TimePassesCallbacks, which would supposedly be run after those queries?!

TimePassesCallbacks enables path trimming very early on. In fact the enabling of it is what causes the panic unless you wrap the printing in either the macro that disables path trimming temporarily or the macro that explicitly allows trimming.

[b-naber]

But does rustc actually do this? I don't understand why this ICE isn't hit in rustc then.

[bjorn3]

Rustc makes sure to wrap all relevant formatting outside of the error path in either of the two macros.

[b-naber]

Ok thanks, I didn't know how these macros worked and where they were used, but I'm still confused about why the stable-mir driver then ICEs. It just calls the rust driver itself, so all those macro calls should be made via the stable mir driver too. I was assuming that rustc must be doing something else that influences how paths are printed before calling the driver, but it doesn't as far as I can tell.

It's also not the after_analysis callback that is to blame for the ICE; I removed that and it still ICEs.

[b-naber]

I rebased my branch and downloaded the rust repo in the script again, and now I don't get any ICEs due to trimmed paths anymore. Hm weird, but I won't complain. I still get some failing tests in the ui suite though:

• some due to the linker not finding the rust_test_helper binary (not sure why)
• error loading target specification: could not find specification for target \"(none)\ (this might be due to my compiletest args, see below for the script):
  command: env -u RUSTC_LOG_COLOR RUSTC_ICE=\"0\" RUST_BACKTRACE=\"short\" \"/tmp/smir/bin/test-drive\" \"/tmp/rustc/tests/ui/attributes/main-removed-2/auxiliary/tokyo.rs\" \"-Zthreads=1\" \"-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX\" \"-Ztranslate-remapped-path-to-local-path=no\" \"-Z\" \"ignore-directory-in-diagnostics-source-blocks=/home/gh-b-naber/.cargo\" \"-Z\" \"ignore-directory-in-diagnostics-source-blocks=/tmp/rustc/vendor\" \"--sysroot\" \"/home/gh-b-naber/rust/build/aarch64-unknown-linux-gnu/stage2\" \"--target=(none)\" \"--check-cfg\" \"cfg(test,FALSE)\" \"--error-format\" \"json\" \"--json\" \"future-incompat\" \"-Ccodegen-units=1\" \"-Zui-testing\" \"-Zdeduplicate-diagnostics=no\" \"-Zwrite-long-types-to-disk=no\" \"-Cstrip=debuginfo\" \"-C\" \"prefer-dynamic\" \"--out-dir\" \"/tmp/rustc/build/aarch64-unknown-linux-gnu/stage1/tests/ui/attributes/main-removed-2/main/auxiliary\" \"-A\" \"internal_features\" \"--smir-check\" \"--crate-type\" \"proc-macro\" \"--extern\" \"proc_macro\" \"-L\" \"/tmp/rustc/build/aarch64-unknown-linux-gnu/stage1/tests/ui/attributes/main-removed-2/main/auxiliary\"
• some path errors where $DIR isn't used, but we get something like ../rust/tests/ui/associated-type-bounds/return-type-notation/path-non-param-qself.rs:11:5:
  command: env -u RUSTC_LOG_COLOR RUSTC_ICE=\"0\" RUST_BACKTRACE=\"short\" \"/tmp/smir/bin/test-drive\" \"/tmp/rustc/tests/ui/associated-type-bounds/return-type-notation/path-non-param-qself.rs\" \"-Zthreads=1\" \"-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX\" \"-Ztranslate-remapped-path-to-local-path=no\" \"-Z\" \"ignore-directory-in-diagnostics-source-blocks=/home/gh-b-naber/.cargo\" \"-Z\" \"ignore-directory-in-diagnostics-source-blocks=/tmp/rustc/vendor\" \"--sysroot\" \"/home/gh-b-naber/rust/build/aarch64-unknown-linux-gnu/stage2\" \"--target=aarch64-unknown-linux-gnu\" \"--check-cfg\" \"cfg(test,FALSE)\" \"--error-format\" \"json\" \"--json\" \"future-incompat\" \"-Ccodegen-units=1\" \"-Zui-testing\" \"-Zdeduplicate-diagnostics=no\" \"-Zwrite-long-types-to-disk=no\" \"-Cstrip=debuginfo\" \"--emit\" \"metadata\" \"-C\" \"prefer-dynamic\" \"--out-dir\" \"/tmp/rustc/build/aarch64-unknown-linux-gnu/stage1/tests/ui/associated-type-bounds/return-type-notation/path-non-param-qself\" \"-A\" \"unused\" \"-A\" \"internal_features\" \"--smir-check\"\
• some ICEs, though all the ones I've checked also occur with the nightly toolchain.

test-failures.txt

The script for running the test suite is:

#!/usr/bin/env bash

# Run rustc test suites using our test driver using nightly.
# This script leverages the rustc's repo compiletest crate.
#
# The suites configuration should match:
# https://github.com/rust-lang/rust/blob/master/src/bootstrap/test.rs

set -e
set -u
export RUST_BACKTRACE=1

# Location of a rust repository. Clone one if path doesn't exist.
RUST_REPO="${RUST_REPO:-"/tmp/rustc"}"

# Where we will store the SMIR tools (Optional).
TOOLS_BIN="${TOOLS_BIN:-"/tmp/smir/bin"}"

# Assume we are inside SMIR repository
SMIR_PATH=$(git rev-parse --show-toplevel)

# Set the toolchain to be used in this script
REPO_TOOLCHAIN=$(rustup show active-toolchain | (read toolchain _; echo $toolchain))
TOOLCHAIN="${TOOLCHAIN:-${REPO_TOOLCHAIN}}"

# Build stable_mir tools
function build_smir_tools() {
  pushd "${SMIR_PATH}"
  cargo +${TOOLCHAIN} build -Z unstable-options --out-dir "${TOOLS_BIN}"
  export PATH="${TOOLS_BIN}":"${PATH}"
}

# Set up rustc repository
function setup_rustc_repo() {
  if [[ ! -e "${RUST_REPO}" ]]; then
    mkdir -p "$(dirname ${RUST_REPO})"
    git clone -b master https://github.com/rust-lang/rust.git "${RUST_REPO}"
    pushd "${RUST_REPO}"
    commit="$(rustc +${TOOLCHAIN} -vV | awk '/^commit-hash/ { print $2 }')"
    if [[ "${commit}" != "unknown" ]]; then
       # For custom toolchain, this may return "unknown". Skip this step if that's the case.
       # In that case, we will use the HEAD of the main branch.
      git checkout "${commit}"
    fi
    git submodule init -- "library/stdarch"
    git submodule update
  else
    echo "rust repo already exists"
    pushd "${RUST_REPO}"
  fi
}

function run_tests() {
  # Run the following suite configuration for now (test suite + mode)
  SUITES=(
    #"codegen codegen"
    #"codegen-units codegen-units"
    # -- The suites below are failing because of fully qualified paths for standard library
    # E.g.:
    # -    _10 = _eprint(move _11) -> [return: bb6, unwind unreachable];
    # +    _10 = std::io::_eprint(move _11) -> [return: bb6, unwind unreachable];
    #
    "ui ui"
    #"mir-opt mir-opt"
    #"pretty pretty" -- 2 failing tests
  )

  SYSROOT=$(rustc +${TOOLCHAIN} --print sysroot)
  PY_PATH=$(type -P python3)
  HOST=$(rustc +${TOOLCHAIN} -vV | awk '/^host/ { print $2 }')
  FILE_CHECK="$(which FileCheck-12 || which FileCheck-13 || which FileCheck-14)"

  echo "#---------- Variables -------------"
  echo "RUST_REPO: ${RUST_REPO}"
  echo "TOOLS_BIN: ${TOOLS_BIN}"
  echo "TOOLCHAIN: ${TOOLCHAIN}"
  echo "SYSROOT: ${SYSROOT}"
  echo "FILE_CHECK: ${FILE_CHECK}"
  echo "-----------------------------------"

  for suite_cfg in "${SUITES[@]}"; do
    # Hack to work on older bash like the ones on MacOS.
    suite_pair=($suite_cfg)
    suite=${suite_pair[0]}
    mode=${suite_pair[1]}

    echo "#### Running suite: ${suite} mode: ${mode}"
    cargo +${TOOLCHAIN} run -p compiletest -- \
      --compile-lib-path="${SYSROOT}/lib" \
      --run-lib-path="${SYSROOT}/lib"\
      --python="${PY_PATH}" \
      --rustc-path="${TOOLS_BIN}/test-drive" \
      --mode=${mode} \
      --suite="${suite}" \
      --src-root="${RUST_REPO}" \
      --src-test-suite-root="${RUST_REPO}/tests/${suite}" \
      --build-root="$(RUST_REPO)/build/${HOST}/stage1" \
      --build-test-suite-root="$(RUST_REPO)/build/${HOST}/stage1/tests/${suite}" \
      --sysroot-base="$SYSROOT" \
      --stage-id=stage1-${HOST} \
      --stage=1 \
      --cc= \
      --cxx= \
      --cflags= \
      --cxxflags= \
      --llvm-components= \
      --android-cross-path= \
      --target=${HOST} \
      --llvm-filecheck="${FILE_CHECK}" \
      --channel=nightly \
      --git-merge-commit-email="" \
      --minicore-path="${RUST_REPO}/tests/auxiliary/minicore.rs" \
      --nightly-branch="main" \
      --target-rustcflags="--smir-check" \
      --host-rustcflags="--smir-check"
  done
}

build_smir_tools
setup_rustc_repo
run_tests

The trimmed path issue is solved though. @celinval Do you think we should install the config callback that sets the trimmed_def_paths flag in the existing driver or should we create a new stable-mir driver for the test-driver?

[celinval]

Would the callback be exposed to the user at all?

[b-naber]

I think it might force the user to use the with_no_trimmed_paths macro in the after_analysis callback, if they ever create strings there.