chore: Enable Comet shuffle for Spark core-1 test

[viirya]

Which issue does this PR close?

Closes #930.

Rationale for this change

We disabled Comet shuffle for Spark core-1 test because negative ref count issue reported by Java Arrow. After fixing memory issues and some changes to more follow C Data interface, we may re-enable Comet shuffle for Spark core-1 test in CI.

What changes are included in this PR?

How are these changes tested?

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 34.15%. Comparing base (033fe6f) to head (9fb03f1).
Report is 13 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #924      +/-   ##
============================================
+ Coverage     34.03%   34.15%   +0.11%     
+ Complexity      883      882       -1     
============================================
  Files           113      112       -1     
  Lines         43170    43237      +67     
  Branches       9516     9562      +46     
============================================
+ Hits          14693    14766      +73     
+ Misses        25471    25455      -16     
- Partials       3006     3016      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[viirya]

Seems negative ref count issue was fixed by previous commits revising array import/export.

[comphead]

lgtm thanks @viirya
1 minor why we have RUST_BACKTRACE enabled for ansi workflow test and disabled for regular one?

[viirya]

lgtm thanks @viirya 1 minor why we have RUST_BACKTRACE enabled for ansi workflow test and disabled for regular one?

Thanks @comphead

I'm not sure about it. Maybe we have added it for debugging ansi workflow before?