-
Notifications
You must be signed in to change notification settings - Fork 71
Closed
Labels
Type: bugSomething isn't workingSomething isn't workingType: enhancementNew feature or requestNew feature or request
Milestone
Description
Describe the enhancement requested
Our product, which uses Gandiva, was encountering errors from LLVM during intense query processing stress tests. I traced the issue back to the JIT upgrade (apache/arrow#39098), but it seems it only started occurring then because the new JIT engine requires unique symbol names unlike the old engine.
The error from llvm is: Duplicate definition of symbol 'expr_0_0'.
After updating the llvm version to 18 (apache/arrow#45114) you can see this error occasionally when running the Gandiva ProjectorTest.
Failed to make LLVM module due to Failed to add IR module to LLJIT: Duplicate definition of symbol 'expr_0_0'
at org.apache.arrow.gandiva.evaluator.JniWrapper.buildProjector(Native Method)
at org.apache.arrow.gandiva.evaluator.Projector.make(Projector.java:206)
at org.apache.arrow.gandiva.evaluator.Projector.make(Projector.java:70)
at org.apache.arrow.gandiva.evaluator.ProjectorTest.lambda$testMakeProjectorParallel$1(ProjectorTest.java:138)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[WARNING] Tests run: 44, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 6.656 s -- in org.apache.arrow.gandiva.evaluator.ProjectorTest
Adding synchronization the the make and evaluate methods in gandiva/src/main/java/org/apache/arrow/gandiva/evaluator/Projector.java will fix the issue.
Metadata
Metadata
Assignees
Labels
Type: bugSomething isn't workingSomething isn't workingType: enhancementNew feature or requestNew feature or request