Working zstd support!

[kylebarron]

Needs this unreleased commit from zstd-rs, which was merged on Jan 28, and didn't make it into version 0.10.0.

It also needs default-features = false on zstd. That will be a change necessary in arrow-rs. It looks like the only string of zstd:: in arrow-rs is in parquet/src/compression.rs, and that seems to work fine with default-features=false. So this update in Arrow shouldn't be an issue.

Also needs to use the non-Apple clang for compiling. Ref rust-bitcoin/rust-secp256k1#283 (comment) and rust-bitcoin/rust-secp256k1#283 (comment) and the upstream zstd issue gyscos/zstd-rs#93. When only PATH was updated but not CC or AR, I got error: failed to build archive: section too large.

Repro:

brew install llvm
export PATH="/usr/local/opt/llvm/bin/:$PATH"
export CC=/usr/local/opt/llvm/bin/clang
export AR=/usr/local/opt/llvm/bin/llvm-ar

[kylebarron]

Closing, as the updated zstd version is in progress upstream. The next versions of the parquet and parquet2 crates will hopefully depend on 0.11.1+ and use default-features = false.