Support client-side encryption

[xanderbailey]

What's the feature are you trying to implement?

Iceberg spec supports client-side encryption and this is largely fully supported in the Java client but we're lagging behind in rust and I think we should change that.

Iceberg docs

Willingness to contribute

I would be willing to contribute to this feature with guidance from the Iceberg Rust community

[mbutrovich]

Tagging relevant discussion on Arrow-rs (the underlying Parquet reader) and DataFusion since @ggershinksy (who drove a lot of the Parquet Modular Encryption and Iceberg table encryption work) was active in the discussion there:

apache/arrow-rs#6637
apache/arrow-rs#7111
apache/arrow-rs#7387
apache/datafusion#15216

I think this can be helpful context. I'll also tag the work we did with Comet to enable PME with arbitrary Spark KMS via JNI:

apache/datafusion-comet#2447