GH-55: [Gandiva] Re-enable tests

[lriggs]

Fixes GH-55.

Rationale for this change

I fixed the linking error here: apache/arrow#45114

What changes are included in this PR?

Reenabling some disabled tests.

Are these changes tested?

Yes, they are tests.

Are there any user-facing changes?

No.

[lriggs]

How do you specify the arrow version used to build/test with arrow-java (so that I know it has my fix)? Does it always use the latest?

[kou]

We use the latest release for push/pull request:

arrow-java/.github/workflows/rc.yml

Lines 110 to 113 in a5b8604

	- name: Download the latest Apache Arrow C++
	if: github.event_name != 'schedule'
	run: |
	ci/scripts/download_cpp.sh

We use the main branch for schedule:

arrow-java/.github/workflows/rc.yml

Lines 114 to 119 in a5b8604

	- name: Checkout Apache Arrow C++
	if: github.event_name == 'schedule'
	uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
	with:
	repository: apache/arrow
	path: arrow

We need to update

arrow-java/.env

Line 57 in a5b8604

VCPKG="943c5ef1c8f6b5e6ced092b242c8299caae2ff01" # 2024.04.26 Release

to use vcpkg configuration introduced by apache/arrow#45114 .

[lriggs]

Thanks. I updated vcpkg as well now.

[lidavidm]

Looks like one more minor CI nit:

  Error:  /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/FilterProjectTest.java:37:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]

[lidavidm]

There's others - are you able to run the build locally at all to test?

   Error:  /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/FilterTest.java:37:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]
  Error:  /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/ProjectorDecimalTest.java:45:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]

[lriggs]

Now its failing with "Gandiva Failed to make LLVM module due to Could not create LLJIT instance: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]" which is what should be fixed in my other change in apache/arrow.

Seems to be using the 18.2 jars: /home/runner/.m2/repository/org/apache/arrow/arrow-avro/18.2.0/

[lidavidm]

Ah, the CI here uses the latest released version of the C++ libraries, not the development version...

[lidavidm]

We could either make the skips conditional on the version of the C++ code somehow, or otherwise just always test against the dev release?

[kou]

C++ version must not be related.
apache/arrow#45114 just updated LLVM version.

[lidavidm]

But we're building 19.0.0, not main: https://github.com/apache/arrow-java/actions/runs/13298859368/job/37137015507?pr=595

[kou]

Ah, I thought that updating VCPKG in .env is enough to use LLVM 18 but we refer ci/vcpkg in apache/arrow here...:

arrow-java/ci/docker/vcpkg-jni.dockerfile

Line 26 in 34e2b08

--x-manifest-root=/arrow/ci/vcpkg \

The log shows that we use LLVM 17:

https://github.com/apache/arrow-java/actions/runs/13298859368/job/37137015507?pr=595#step:10:1663

  -- Found LLVMAlt: 17.0.2

OK. Let's use main for now:

diff --git a/.github/workflows/rc.yml b/.github/workflows/rc.yml
index faad5983..f2666a31 100644
--- a/.github/workflows/rc.yml
+++ b/.github/workflows/rc.yml
@@ -299,13 +299,13 @@ jobs:
         shell: bash
         run: |
           tar -xf apache-arrow-java-*.tar.gz --strip-components=1
-      - name: Download the latest Apache Arrow C++
-        if: github.event_name != 'schedule'
-        shell: bash
-        run: |
-          ci/scripts/download_cpp.sh
+      # - name: Download the latest Apache Arrow C++
+      #   if: github.event_name != 'schedule'
+      #   shell: bash
+      #   run: |
+      #     ci/scripts/download_cpp.sh
       - name: Checkout Apache Arrow C++
-        if: github.event_name == 'schedule'
+        # if: github.event_name == 'schedule'
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           repository: apache/arrow

[lidavidm]

Ah, I see what you meant by the original comment now...

[lriggs]

I'll try your suggested change @kou

[lriggs]

Latest failures were because I only updated the windows build in rc.yml. Just fixed that for mac and linux.

[kou]

Oh, sorry. It's my fault...
We just need to use main only for Linux because Linux build only uses vcpkg to install LLVM.

[lriggs]

Strange, it still seems to be using the old vcpkg.

Does arrow-java need something like this? https://github.com/apache/arrow/blob/main/ci/scripts/install_vcpkg.sh

2025-02-14T00:07:03.2003955Z CMake Warning at /opt/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package):
2025-02-14T00:07:03.2004967Z   Could not find a configuration file for package "LLVM" that is compatible
2025-02-14T00:07:03.2005618Z   with requested version "19.1".

....

2025-02-14T00:07:03.2600773Z -- Found LLVMAlt: 17.0.2
2025-02-14T00:07:03.2601530Z -- Using LLVMConfig.cmake in: /opt/vcpkg/installed/amd64-linux-static-release/share/llvm

[kou]

Ah, we need to expire a cache when we change vcpkg version. Could you try this?

diff --git a/compose.yaml b/compose.yaml
index 58a19676..b125c3c9 100644
--- a/compose.yaml
+++ b/compose.yaml
@@ -92,12 +92,12 @@ services:
     # Usage:
     #   docker compose build vcpkg-jni
     #   docker compose run vcpkg-jni
-    image: ${REPO}:${ARCH}-vcpkg-jni
+    image: ${REPO}:${ARCH}-vcpkg-jni-${VCPKG}
     build:
       context: .
       dockerfile: ci/docker/vcpkg-jni.dockerfile
       cache_from:
-        - ${REPO}:${ARCH}-vcpkg-jni
+        - ${REPO}:${ARCH}-vcpkg-jni-${VCPKG}
       args:
         base: ${ARROW_REPO}:${ARCH}-python-${PYTHON}-wheel-manylinux-2014-vcpkg-${VCPKG}
     volumes:

[lriggs]

Ok, just pushed a change to expire the cache. Hopefully it works.

[lidavidm]

Looks like that worked!