Skip to content

Commit f539d92

Browse files
[ci] Write test results to unique file names (#113160)
In this patch I'm using a new lit option so that the pipeline writes many results files, one for each time lit is run: ``` --use-unique-output-file-name When enabled, lit will add a unique element to the output file name, before the extension. For example "results.xml" will become "results.<something>.xml". The "<something>" is not ordered in any way and is chosen so that existing files are not overwritten. [Default: Off] ``` (I added this to lit recently) Alternatives were considered: * mkfifo - does not work on bash for Windows. * tail -f - does not print full content on file truncation * lit wrapper script - more complication than using an option to lit itself * ninja/mv file/ninja/mv file etc - lots of changes needed to make the scripts build each target separately And after feedback I decided that using an option to lit itself is the cleanest way to go. It can be removed when we no longer need it. If I run the Linux build after this change: ``` $ bash ./.ci/monolithic-linux.sh "clang;lldb;lld" "check-lldb-shell check-lld" "libcxx;libcxxabi" "check-libcxx check-libcxxabi" ``` I get multiple test result files. In my case some tests fail so runtimes aren't checked, but all projects are so there is 1 file for lldb and one for lld: ``` $ ls build/*.xml build/test-results.klc82utf.xml build/test-results.majylh73.xml ``` This change just collects the XML files as artifacts. Once I know that's working, I can set up test reporting to make a summary of them.
1 parent 1b63f47 commit f539d92

File tree

3 files changed

+12
-7
lines changed

3 files changed

+12
-7
lines changed

.ci/generate-buildkite-pipeline-premerge

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -272,7 +272,7 @@ if [[ "${linux_projects}" != "" ]]; then
272272
artifact_paths:
273273
- 'artifacts/**/*'
274274
- '*_result.json'
275-
- 'build/test-results.xml'
275+
- 'build/test-results.*.xml'
276276
agents: ${LINUX_AGENTS}
277277
retry:
278278
automatic:
@@ -295,7 +295,7 @@ if [[ "${windows_projects}" != "" ]]; then
295295
artifact_paths:
296296
- 'artifacts/**/*'
297297
- '*_result.json'
298-
- 'build/test-results.xml'
298+
- 'build/test-results.*.xml'
299299
agents: ${WINDOWS_AGENTS}
300300
retry:
301301
automatic:

.ci/monolithic-linux.sh

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@ trap show-stats EXIT
3737
projects="${1}"
3838
targets="${2}"
3939

40+
lit_args="-v --xunit-xml-output ${BUILD_DIR}/test-results.xml --use-unique-output-file-name --timeout=1200 --time-tests"
41+
4042
echo "--- cmake"
4143
pip install -q -r "${MONOREPO_ROOT}"/mlir/python/requirements.txt
4244
pip install -q -r "${MONOREPO_ROOT}"/lldb/test/requirements.txt
@@ -47,7 +49,7 @@ cmake -S "${MONOREPO_ROOT}"/llvm -B "${BUILD_DIR}" \
4749
-D LLVM_ENABLE_ASSERTIONS=ON \
4850
-D LLVM_BUILD_EXAMPLES=ON \
4951
-D COMPILER_RT_BUILD_LIBFUZZER=OFF \
50-
-D LLVM_LIT_ARGS="-v --xunit-xml-output ${BUILD_DIR}/test-results.xml --timeout=1200 --time-tests" \
52+
-D LLVM_LIT_ARGS="${lit_args}" \
5153
-D LLVM_ENABLE_LLD=ON \
5254
-D CMAKE_CXX_FLAGS=-gmlt \
5355
-D LLVM_CCACHE_BUILD=ON \
@@ -87,7 +89,8 @@ if [[ "${runtimes}" != "" ]]; then
8789
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
8890
-D CMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
8991
-D LIBCXX_TEST_PARAMS="std=c++03" \
90-
-D LIBCXXABI_TEST_PARAMS="std=c++03"
92+
-D LIBCXXABI_TEST_PARAMS="std=c++03" \
93+
-D LLVM_LIT_ARGS="${lit_args}"
9194

9295
echo "--- ninja runtimes C++03"
9396

@@ -104,7 +107,8 @@ if [[ "${runtimes}" != "" ]]; then
104107
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
105108
-D CMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
106109
-D LIBCXX_TEST_PARAMS="std=c++26" \
107-
-D LIBCXXABI_TEST_PARAMS="std=c++26"
110+
-D LIBCXXABI_TEST_PARAMS="std=c++26" \
111+
-D LLVM_LIT_ARGS="${lit_args}"
108112

109113
echo "--- ninja runtimes C++26"
110114

@@ -121,7 +125,8 @@ if [[ "${runtimes}" != "" ]]; then
121125
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
122126
-D CMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
123127
-D LIBCXX_TEST_PARAMS="enable_modules=clang" \
124-
-D LIBCXXABI_TEST_PARAMS="enable_modules=clang"
128+
-D LIBCXXABI_TEST_PARAMS="enable_modules=clang" \
129+
-D LLVM_LIT_ARGS="${lit_args}"
125130

126131
echo "--- ninja runtimes clang modules"
127132

.ci/monolithic-windows.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ cmake -S "${MONOREPO_ROOT}"/llvm -B "${BUILD_DIR}" \
5353
-D LLVM_ENABLE_ASSERTIONS=ON \
5454
-D LLVM_BUILD_EXAMPLES=ON \
5555
-D COMPILER_RT_BUILD_LIBFUZZER=OFF \
56-
-D LLVM_LIT_ARGS="-v --xunit-xml-output ${BUILD_DIR}/test-results.xml --timeout=1200 --time-tests" \
56+
-D LLVM_LIT_ARGS="-v --xunit-xml-output ${BUILD_DIR}/test-results.xml --use-unique-output-file-name --timeout=1200 --time-tests" \
5757
-D COMPILER_RT_BUILD_ORC=OFF \
5858
-D CMAKE_C_COMPILER_LAUNCHER=sccache \
5959
-D CMAKE_CXX_COMPILER_LAUNCHER=sccache \

0 commit comments

Comments
 (0)