Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full path names are used in several unittests instead of the binary name. Fix up the testcase failures #107974

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

HighW4y2H3ll
Copy link
Contributor

@HighW4y2H3ll HighW4y2H3ll commented Sep 10, 2024

Encountered several testcase failures when running ninja check-all. It was due to the full path name were shown in the error message instead of the binary name, and therefore causing the check string mismatch.

The machine was running CentOS 9 with binfmt_misc setup that uses qemu-aarch64 (8.1.2). Built and ran the unittest as aarch64 host (through qemu user).

@llvmbot
Copy link
Collaborator

llvmbot commented Sep 10, 2024

@llvm/pr-subscribers-testing-tools

@llvm/pr-subscribers-clang-tools-extra

Author: None (HighW4y2H3ll)

Changes

Encountered several testcase failures when running ninja check-all. It was due to the full path name were shown in the error message instead of the binary name, and therefore causing the check string mismatch.


Full diff: https://github.com/llvm/llvm-project/pull/107974.diff

4 Files Affected:

  • (modified) clang-tools-extra/clangd/test/log.test (+1-1)
  • (modified) clang-tools-extra/test/clang-query/invalid-command-line.cpp (+1-1)
  • (modified) clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp (+1-1)
  • (modified) llvm/utils/lit/tests/shtest-output-printing.py (+1-1)
diff --git a/clang-tools-extra/clangd/test/log.test b/clang-tools-extra/clangd/test/log.test
index 7a53d361ddde5f..5cc871972f9892 100644
--- a/clang-tools-extra/clangd/test/log.test
+++ b/clang-tools-extra/clangd/test/log.test
@@ -1,7 +1,7 @@
 # RUN: env CLANGD_FLAGS=-compile-commands-dir=no-such-dir not clangd -lit-test </dev/null 2>&1 >/dev/null | FileCheck %s
 CHECK: I[{{.*}}]{{.*}} clangd version {{.*}}
 CHECK: Working directory: {{.*}}
-CHECK: argv[0]: clangd
+CHECK: argv[0]: {{.*}}clangd
 CHECK: argv[1]: -lit-test
 CHECK: CLANGD_FLAGS: -compile-commands-dir=no-such-dir
 CHECK: E[{{.*}}] Path specified by --compile-commands-dir does not exist.
diff --git a/clang-tools-extra/test/clang-query/invalid-command-line.cpp b/clang-tools-extra/test/clang-query/invalid-command-line.cpp
index e3e8af1d5e7ae0..a66acc8037f7d9 100644
--- a/clang-tools-extra/test/clang-query/invalid-command-line.cpp
+++ b/clang-tools-extra/test/clang-query/invalid-command-line.cpp
@@ -1,4 +1,4 @@
 // RUN: not clang-query --invalid-arg 2>&1 | FileCheck %s
 
-// CHECK: error: clang-query{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: 'clang-query{{(\.exe)?}} --help'
+// CHECK: error: clang-query{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: '{{.*}}clang-query{{(\.exe)?}} --help'
 // CHECK-NEXT: clang-query{{(\.exe)?}}: Did you mean '--extra-arg'?
diff --git a/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp b/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp
index c06b09d90004ee..4bdca50af32cac 100644
--- a/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp
+++ b/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp
@@ -1,4 +1,4 @@
 // RUN: not clang-tidy --invalid-arg 2>&1 | FileCheck %s
 
-// CHECK: error: clang-tidy{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: 'clang-tidy{{(\.exe)?}} --help'
+// CHECK: error: clang-tidy{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: '{{.*}}clang-tidy{{(\.exe)?}} --help'
 // CHECK-NEXT: clang-tidy{{(\.exe)?}}: Did you mean '--extra-arg'?
diff --git a/llvm/utils/lit/tests/shtest-output-printing.py b/llvm/utils/lit/tests/shtest-output-printing.py
index 129cff981eb5b2..b9045c3fe520bd 100644
--- a/llvm/utils/lit/tests/shtest-output-printing.py
+++ b/llvm/utils/lit/tests/shtest-output-printing.py
@@ -25,7 +25,7 @@
 #  CHECK-NEXT: not not wc missing-file &> [[FILE:.*]] || true
 #  CHECK-NEXT: # executed command: not not wc missing-file
 #  CHECK-NEXT: # .---redirected output from '[[FILE]]'
-#  CHECK-NEXT: # | wc: {{cannot open missing-file|missing-file.* No such file or directory}}
+#  CHECK-NEXT: # | {{.*}}wc: {{cannot open missing-file|missing-file.* No such file or directory}}
 #  CHECK-NEXT: # `-----------------------------
 #  CHECK-NEXT: # note: command had no output on stdout or stderr
 #  CHECK-NEXT: # error: command failed with exit status: 1

@llvmbot
Copy link
Collaborator

llvmbot commented Sep 10, 2024

@llvm/pr-subscribers-clang-tidy

Author: None (HighW4y2H3ll)

Changes

Encountered several testcase failures when running ninja check-all. It was due to the full path name were shown in the error message instead of the binary name, and therefore causing the check string mismatch.


Full diff: https://github.com/llvm/llvm-project/pull/107974.diff

4 Files Affected:

  • (modified) clang-tools-extra/clangd/test/log.test (+1-1)
  • (modified) clang-tools-extra/test/clang-query/invalid-command-line.cpp (+1-1)
  • (modified) clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp (+1-1)
  • (modified) llvm/utils/lit/tests/shtest-output-printing.py (+1-1)
diff --git a/clang-tools-extra/clangd/test/log.test b/clang-tools-extra/clangd/test/log.test
index 7a53d361ddde5f..5cc871972f9892 100644
--- a/clang-tools-extra/clangd/test/log.test
+++ b/clang-tools-extra/clangd/test/log.test
@@ -1,7 +1,7 @@
 # RUN: env CLANGD_FLAGS=-compile-commands-dir=no-such-dir not clangd -lit-test </dev/null 2>&1 >/dev/null | FileCheck %s
 CHECK: I[{{.*}}]{{.*}} clangd version {{.*}}
 CHECK: Working directory: {{.*}}
-CHECK: argv[0]: clangd
+CHECK: argv[0]: {{.*}}clangd
 CHECK: argv[1]: -lit-test
 CHECK: CLANGD_FLAGS: -compile-commands-dir=no-such-dir
 CHECK: E[{{.*}}] Path specified by --compile-commands-dir does not exist.
diff --git a/clang-tools-extra/test/clang-query/invalid-command-line.cpp b/clang-tools-extra/test/clang-query/invalid-command-line.cpp
index e3e8af1d5e7ae0..a66acc8037f7d9 100644
--- a/clang-tools-extra/test/clang-query/invalid-command-line.cpp
+++ b/clang-tools-extra/test/clang-query/invalid-command-line.cpp
@@ -1,4 +1,4 @@
 // RUN: not clang-query --invalid-arg 2>&1 | FileCheck %s
 
-// CHECK: error: clang-query{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: 'clang-query{{(\.exe)?}} --help'
+// CHECK: error: clang-query{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: '{{.*}}clang-query{{(\.exe)?}} --help'
 // CHECK-NEXT: clang-query{{(\.exe)?}}: Did you mean '--extra-arg'?
diff --git a/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp b/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp
index c06b09d90004ee..4bdca50af32cac 100644
--- a/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp
+++ b/clang-tools-extra/test/clang-tidy/infrastructure/invalid-command-line.cpp
@@ -1,4 +1,4 @@
 // RUN: not clang-tidy --invalid-arg 2>&1 | FileCheck %s
 
-// CHECK: error: clang-tidy{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: 'clang-tidy{{(\.exe)?}} --help'
+// CHECK: error: clang-tidy{{(\.exe)?}}: Unknown command line argument '--invalid-arg'.  Try: '{{.*}}clang-tidy{{(\.exe)?}} --help'
 // CHECK-NEXT: clang-tidy{{(\.exe)?}}: Did you mean '--extra-arg'?
diff --git a/llvm/utils/lit/tests/shtest-output-printing.py b/llvm/utils/lit/tests/shtest-output-printing.py
index 129cff981eb5b2..b9045c3fe520bd 100644
--- a/llvm/utils/lit/tests/shtest-output-printing.py
+++ b/llvm/utils/lit/tests/shtest-output-printing.py
@@ -25,7 +25,7 @@
 #  CHECK-NEXT: not not wc missing-file &> [[FILE:.*]] || true
 #  CHECK-NEXT: # executed command: not not wc missing-file
 #  CHECK-NEXT: # .---redirected output from '[[FILE]]'
-#  CHECK-NEXT: # | wc: {{cannot open missing-file|missing-file.* No such file or directory}}
+#  CHECK-NEXT: # | {{.*}}wc: {{cannot open missing-file|missing-file.* No such file or directory}}
 #  CHECK-NEXT: # `-----------------------------
 #  CHECK-NEXT: # note: command had no output on stdout or stderr
 #  CHECK-NEXT: # error: command failed with exit status: 1

@MatzeB
Copy link
Contributor

MatzeB commented Sep 10, 2024

Please provide additional context in your summary on how this happens.

My understanding is that this is for a setup with linux binfmt-misc + qemu running aarch64 binaries on an x86 system. The fact that we end up with absolute pathnames in argv[0] may be a bug in qemu or binfmt-misc or maybe it's not specified in posix etc. and any behavior is allowed (I don't really know).

Given this only affects 4 tests in all of llvm-project I think it is fine to just adapt the tests (even if it would turn out incorrect behavior).

So LGTM but maybe wait a couple days in case there are different opinions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants