Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: pin ubuntu runner to 20.04 to avoid sanitizer failures #767

Merged
merged 4 commits into from
Mar 18, 2024

Conversation

hellkite500
Copy link
Member

Recently tests are failing with AddressSanitizer:DEADLYSIGNAL. A ctest timeout occurs and causes the test suite to fail. This does NOT occur when actually running the tests, but instead during the build/linking step. In the test CMakeLists, there is a call to

gtest_discover_tests

which defers test discovery until runtime instead of scanning source files. In effect, the causes the test executable to run with the flag --gtest_list_tests.
The cmake error points to this call as failing:

CMake Error at /usr/local/share/cmake-3.28/Modules/GoogleTestAddTests.cmake:112 (message):
2024-03-15T22:38:17.3140582Z   Error running test executable.
2024-03-15T22:38:17.3140590Z 
2024-03-15T22:38:17.3140878Z     Path: '/home/runner/work/ngen/ngen/cmake_build/test/test_bmi_c'
2024-03-15T22:38:17.3141033Z     Result: Process terminated due to timeout
2024-03-15T22:38:17.3141126Z     Output:
2024-03-15T22:38:17.3141215Z       
2024-03-15T22:38:17.3141230Z 
2024-03-15T22:38:17.3141350Z Call Stack (most recent call first):
2024-03-15T22:38:17.3141791Z   /usr/local/share/cmake-3.28/Modules/GoogleTestAddTests.cmake:225 (gtest_discover_tests_impl)

It appears that some combination of google test, ubuntu, and VM/Containers is causes the address sanitizer to trigger an infinite loop which causes the timeout.

See quantumlib/Stim#717 and google/googletest#4491

Changes

  • Change from ubuntu-latest to ubuntu-20.04 in github CI runners

Target Environment support

  • GitHub CI ubuntu runner

@hellkite500
Copy link
Member Author

Not sure why they are just now causing the CI to fail considering the change has been merged for 7 months, but the pedantic errors are causing failures based on a lot of extra ; throughout the code. I'll trace these down and push that fix as a commit to this PR as well.

@PhilMiller PhilMiller linked an issue Mar 18, 2024 that may be closed by this pull request
@hellkite500 hellkite500 force-pushed the ci-sanitizer-failure branch 3 times, most recently from f89490d to a0cacb4 Compare March 18, 2024 20:34
@hellkite500 hellkite500 merged commit 8e0a95c into master Mar 18, 2024
19 checks passed
@hellkite500 hellkite500 deleted the ci-sanitizer-failure branch March 18, 2024 22:03
@PhilMiller
Copy link
Contributor

Github staff integrated a workaround in their CI runner image: actions/runner-images#9491 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AddressSanitizer (ASan) DEADLYSIGNAL in Github CI actions
2 participants