Can't publish datafusion-spark crate due to error

[alamb]

Describe the bug

Filing a ticket for something @xudong963 hit today (I can't remember where this was mentioned):

When trying to publish the datafusion-spark crate it generates the following error

error[E0432]: unresolved import `datafusion_functions::crypto::basic`
  --> datafusion/spark/src/function/hash/sha2.rs:29:39
   |
29 | pub use datafusion_functions::crypto::basic::{sha224, sha256, sha384, sha512};
   |                                       ^^^^^ could not find `basic` in `crypto`

To Reproduce

Interestingly, when I was working on some documentation as part of

• [datafusion-spark] Example of using Spark compatible function library #15915

I also hit the same error.

Expected behavior

No response

Additional context

No response

[alamb]

I found I could fix the compilation error by applying this patch

diff --git a/datafusion/spark/Cargo.toml b/datafusion/spark/Cargo.toml
index 1ded8c40a..caebcbd2e 100644
--- a/datafusion/spark/Cargo.toml
+++ b/datafusion/spark/Cargo.toml
@@ -41,6 +41,6 @@ datafusion-catalog = { workspace = true }
 datafusion-common = { workspace = true }
 datafusion-execution = { workspace = true }
 datafusion-expr = { workspace = true }
-datafusion-functions = { workspace = true }
+datafusion-functions = { workspace = true , features = ["crypto_expressions"]}
 datafusion-macros = { workspace = true }
 log = { workspace = true }
diff --git a/datafusion/spark/src/lib.rs b/datafusion/spark/src/lib.rs
index 1fe5b6eca..04e2038b1 100644
--- a/datafusion/spark/src/lib.rs
+++ b/datafusion/spark/src/lib.rs

[alamb]

I have a PR to fix it here (found while writing tests):

• [datafusion-spark] Example of using Spark compatible function library #16384

[xudong963]

After this is fixed, we can publish sqllogictest, because it depends on spark ♾

cd datafusion/sqllogictest && cargo publish --allow-dirty
    Updating crates.io index
   Packaging datafusion-sqllogictest v48.0.0 (/Users/xudong/opensource/datafusion/datafusion/sqllogictest)
    Updating crates.io index
error: failed to prepare local package for uploading

Caused by:
  no matching package named `datafusion-spark` found
  location searched: crates.io index
  required by package `datafusion-sqllogictest v48.0.0 (/Users/xudong/opensource/datafusion/datafusion/sqllogictest)`

[alamb]

I manually updated the cargo file (applied the patch above) and ran cargo publish to get this to publish at 48.0.0:

• https://crates.io/crates/datafusion-spark/48.0.0

[xudong963]

Nice, just published datafusion-sqllogictest by fixing the files locally.

All crates have been published!!!

[alamb]

🎉