GH-48277: [C++][Parquet] unpack with shuffle algorithm

[AntoinePrv]

Rationale for this change

What changes are included in this PR?

• Add a new method for building unpacking kernels.
  The constexpr code generation creates a kernel appropriate for a given input/output bit width and simd size.
• I have included a number of xsimd fallback that have been merged upstream.
• I have run extensive benchmarks and re-dispatched among different sizes on specific architectures when it was not performing well.
• The biggest win here is SSE4.2, though AVX2 improves too.
• This is not built/tested for AVX512, though there are not really limitation. Currently the arch detection between all the avx512 is not consistent and sometimes error. I would need to investigate with the upcoming xsimd release.

Are these changes tested?

Yes

Are there any user-facing changes?

No

• GitHub Issue: [C++][Parquet] Better simd unpack algorithm #48277

[github-actions]

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

• Other pull requests
• Contribution Guidelines - Contributing Overview

[github-actions]

⚠️ GitHub issue #48277 has been automatically assigned in GitHub to PR creator.

[AntoinePrv]

@pitrou apart from R-lint, this is looking pretty good.

[pitrou]

@ursabot please benchmark lang=C++

[voltrondatabot]

Benchmark runs are scheduled for commit a4bfe8a. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit a4bfe8a.

There were 37 benchmark results indicating a performance regression:

• Pull Request Run on amd64-c6a-4xlarge-linux at 2025-11-27 19:32:37Z
  • BM_UnpackUint64 (C++) with params=DynamicAligned/47/64, source=cpp-micro, suite=arrow-bpacking-benchmark
  • IsInInt64SmallSet (C++) with params=64, source=cpp-micro, suite=arrow-compute-scalar-set-lookup-benchmark
• and 35 more (see the report linked below)

The full Conbench report has more details.

[AntoinePrv]

@pitrou I'm running this locally, and I made an error when fixing ASAN over-reading problem.
These latest benchmarks are not doing well.

[pitrou]

@ursabot please benchmark lang=C++

[voltrondatabot]

Benchmark runs are scheduled for commit dd3ec0d. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit dd3ec0d.

There were 19 benchmark results indicating a performance regression:

• Pull Request Run on amd64-c6a-4xlarge-linux at 2025-11-28 15:12:47Z
  • BM_UnpackUint32 (C++) with params=DynamicUnaligned/20/64, source=cpp-micro, suite=arrow-bpacking-benchmark
  • BM_DeltaLengthDecodingByteArray (C++) with params=max-string-length:8/batch-size:2048, source=cpp-micro, suite=parquet-encoding-benchmark
• and 17 more (see the report linked below)

The full Conbench report has more details.

[pitrou]

@ursabot please benchmark lang=C++

[voltrondatabot]

Benchmark runs are scheduled for commit 408ef04. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[pitrou]

@ursabot please benchmark lang=C++

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 0 benchmarking runs that have been run so far on PR commit 408ef04.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 0 benchmarking runs that have been run so far on PR commit 408ef04.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.