GH-47620: [CI][C++] Use Ubuntu 24.04 for ASAN UBSAN job #47623

kou · 2025-09-22T22:39:21Z

Rationale for this change

Ubuntu 22.04 ships old Node.js (12). It's too old for Azurite.

What changes are included in this PR?

Use Ubuntu 24.04 instead of 22.04. Ubuntu 24.04 ships newer Node.js (18) that can be used for Azurite.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

GitHub Issue: [C++][CI] arrow-azurefs-test timeouts #47620

github-actions · 2025-09-22T22:39:47Z

⚠️ GitHub issue #47620 has been automatically assigned in GitHub to PR creator.

pitrou · 2025-09-23T07:17:17Z

We have other Ubuntu 22.04 jobs. Should we disable Azure tests on them?

kou · 2025-09-23T07:32:45Z

Yes.

Ubuntu 22.04 ships old Node.js (12). It's too old for Azurite. Ubuntu 24.04 ships newer Node.js (18) that is available for Azurite.

kou · 2025-09-24T12:27:46Z

This is ready.

I think that the Integration job failure is unrelated but it's persisted...

https://github.com/apache/arrow/actions/runs/17970820404/job/51128774114#step:13:1146

  + go install -v ./...
  type_traits_boolean.go:20:2: cannot find package "github.com/apache/arrow-go/v18/arrow/bitutil" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/bitutil (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/bitutil (from $GOPATH)
  datatype_fixedwidth.go:25:2: cannot find package "github.com/apache/arrow-go/v18/arrow/decimal" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/decimal (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/decimal (from $GOPATH)
  datatype_viewheader.go:23:2: cannot find package "github.com/apache/arrow-go/v18/arrow/endian" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/endian (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/endian (from $GOPATH)
  type_traits.go:24:2: cannot find package "github.com/apache/arrow-go/v18/arrow/float16" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/float16 (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/float16 (from $GOPATH)
  datatype.go:24:2: cannot find package "github.com/apache/arrow-go/v18/arrow/internal/debug" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/internal/debug (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/internal/debug (from $GOPATH)
  array.go:22:2: cannot find package "github.com/apache/arrow-go/v18/arrow/memory" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/arrow/memory (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/arrow/memory (from $GOPATH)
  array.go:23:2: cannot find package "github.com/apache/arrow-go/v18/internal/json" in any of:
  	/opt/go/src/github.com/apache/arrow-go/v18/internal/json (from $GOROOT)
  	/go/src/github.com/apache/arrow-go/v18/internal/json (from $GOPATH)
...

pitrou · 2025-09-24T12:32:34Z

I think that the Integration job failure is unrelated but it's persisted...

I see Go failures in the integration test from time to time as well. @zeroshade

pitrou · 2025-09-24T12:33:47Z

ci/scripts/install_azurite.sh

-    azurite_version=latest
+  v12.*)
+    echo "Node.js is too old. We can't install Azurite."
+    exit


Perhaps return an error here?

Suggested change

exit

exit 1

If we return 1 here, docker build is failed on Ubuntu 22.04.

I'll remove this check and remove install_azurite.sh from Ubuntu 22.04 based Dockerfile instead.

pitrou · 2025-09-24T12:34:59Z

@github-actions crossbow submit testubuntu

github-actions · 2025-09-24T12:37:39Z

Revision: 34f3ceb

Submitted crossbow builds: ursacomputing/crossbow @ actions-b7b2cfe8e8

Task	Status
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-r-rhub-ubuntu-release-latest
test-r-ubuntu-22.04
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer
test-ubuntu-24.04-python-3
test-ubuntu-c-glib
test-ubuntu-ruby

kou · 2025-09-25T05:57:36Z

@github-actions crossbow submit testubuntu

github-actions · 2025-09-25T06:01:28Z

Revision: 69a1a35

Submitted crossbow builds: ursacomputing/crossbow @ actions-6d2ba234a9

Task	Status
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-r-rhub-ubuntu-release-latest
test-r-ubuntu-22.04
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer
test-ubuntu-24.04-python-3
test-ubuntu-c-glib
test-ubuntu-ruby

kou · 2025-09-25T09:02:38Z

This is ready.

raulcd · 2025-09-25T09:36:47Z

cpp/src/arrow/filesystem/azurefs_test.cc

-    SetUpInternal(env);
+    auto env_result = AzuriteEnv::GetInstance();
+    if (!env_result.ok()) {
+      GTEST_SKIP() << "Failed to setup: " << env_result.status().ToString();


couldn't this makes us miss tests being skipped due to issues with our environment? We could potentially stop testing Azure and not realize

This may happen but disabling Azure tests manually may be inconvenient...

BTW, we can do it by something like:

diff --git a/ci/scripts/cpp_test.sh b/ci/scripts/cpp_test.sh index 88c06849c8..6b9011d570 100755 --- a/ci/scripts/cpp_test.sh +++ b/ci/scripts/cpp_test.sh @@ -46,6 +46,10 @@ ctest_options=() case "$(uname)" in Linux) n_jobs=$(nproc) + if ! type azurite >/dev/null 2>&1; then + exclude_tests="arrow-azurefs-test" + ctest_options+=(--exclude-regex "${exclude_tests}") + fi ;; Darwin) n_jobs=$(sysctl -n hw.ncpu)

Thanks @kou I think I prefer the second approach so at least we have the safeguard of the macOS cpp job that installs azurite to fail if the install_azurite.sh script starts silently failing again.

OK. I've changed to use the approach.

AzureFS tests are excluded On Ubuntu 22.04:

https://github.com/apache/arrow/actions/runs/18025582047/job/51291868066?pr=47623#step:7:6958

+ ctest --label-regex unittest --output-on-failure --parallel 4 --repeat until-pass:3 --timeout 300 --exclude-regex arrow-azurefs-test

AzureFS tests are executed On Ubuntu 24.04:

https://github.com/apache/arrow/actions/runs/18025582047/job/51291868086?pr=47623#step:7:7607

Start 76: arrow-azurefs-test

conbench-apache-arrow · 2025-09-26T14:23:10Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 41659b8.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.

…#47623) ### Rationale for this change Ubuntu 22.04 ships old Node.js (12). It's too old for Azurite. ### What changes are included in this PR? Use Ubuntu 24.04 instead of 22.04. Ubuntu 24.04 ships newer Node.js (18) that can be used for Azurite. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: apache#47620 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>

kou requested review from assignUser, jonkeane and raulcd as code owners September 22, 2025 22:39

github-actions bot added the awaiting committer review Awaiting committer review label Sep 22, 2025

kou force-pushed the ci-cpp-sanitizer-ubuntu-24.04 branch from 44cca41 to 0ffb9f9 Compare September 23, 2025 03:32

kou force-pushed the ci-cpp-sanitizer-ubuntu-24.04 branch from 0ffb9f9 to 1600bd3 Compare September 23, 2025 13:23

github-actions bot added the Component: C++ label Sep 24, 2025

kou added 4 commits September 24, 2025 17:20

apacheGH-47620: [CI][C++] Use Ubuntu 24.04 for ASAN UBSAN job

3a75b27

Ubuntu 22.04 ships old Node.js (12). It's too old for Azurite. Ubuntu 24.04 ships newer Node.js (18) that is available for Azurite.

Skip

a09ecc7

Fix skip timing

6714254

Add more setup check

568e719

kou force-pushed the ci-cpp-sanitizer-ubuntu-24.04 branch from dd57e6e to 568e719 Compare September 24, 2025 08:21

Remove garbage

34f3ceb

pitrou reviewed Sep 24, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Sep 24, 2025

Don't use install_azurite.sh on Ubuntu 22.04

69a1a35

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 24, 2025

pitrou approved these changes Sep 25, 2025

View reviewed changes

raulcd reviewed Sep 25, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Sep 25, 2025

Skip AzureFS tests in cpp_test.sh not test itself

6912e1c

github-actions bot added awaiting change review Awaiting change review awaiting changes Awaiting changes and removed Component: C++ awaiting changes Awaiting changes awaiting change review Awaiting change review labels Sep 26, 2025

raulcd approved these changes Sep 26, 2025

View reviewed changes

raulcd merged commit 41659b8 into apache:main Sep 26, 2025
37 of 38 checks passed

raulcd removed the awaiting changes Awaiting changes label Sep 26, 2025

raulcd mentioned this pull request Sep 26, 2025

[C++][CI] arrow-azurefs-test timeouts #47620

Closed

github-actions bot added the awaiting merge Awaiting merge label Sep 26, 2025

kou deleted the ci-cpp-sanitizer-ubuntu-24.04 branch September 26, 2025 07:57

kou mentioned this pull request Sep 30, 2025

[CI][Integaration] Building Apache Arrow Go in integration test is flaky #47673

Closed

GH-47620: [CI][C++] Use Ubuntu 24.04 for ASAN UBSAN job #47623

GH-47620: [CI][C++] Use Ubuntu 24.04 for ASAN UBSAN job #47623

Conversation

kou commented Sep 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions bot commented Sep 22, 2025

Uh oh!

pitrou commented Sep 23, 2025

Uh oh!

kou commented Sep 23, 2025

Uh oh!

kou commented Sep 24, 2025

Uh oh!

pitrou commented Sep 24, 2025

Uh oh!

pitrou Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

kou Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

pitrou commented Sep 24, 2025

Uh oh!

github-actions bot commented Sep 24, 2025

Uh oh!

kou commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

kou commented Sep 25, 2025

Uh oh!

raulcd Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

kou Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

raulcd Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

kou Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

kou Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

conbench-apache-arrow bot commented Sep 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kou commented Sep 22, 2025 •

edited by github-actions bot

Loading