Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include Hive and GCS runtime dependencies in main Iceberg jar #32218

Closed
wants to merge 6 commits into from

Conversation

ahmedabu98
Copy link
Contributor

Puts dependencies needed for HiveCatalog and GCS read/write straight into the main beam-sdks-java-io-iceberg jar to make the user experience smoother

Copy link
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@ahmedabu98
Copy link
Contributor Author

The Iceberg integration test workflow is failing here because I changed the gradle command but it doesn't seem to be picking it up here. The workflow passes locally though

@Abacn
Copy link
Contributor

Abacn commented Aug 19, 2024

fyi it is found hive-exec does not support Java9+: #32189. If it affects IcebergIO tests then some adjustments needs to be done on the workflow, similar to what has been done in https://github.com/apache/beam/pull/32190/files#diff-9e3d02467b9b6d6a1e5c3a5ac0d9ce67540e5d61f9f4b52501efe42c9c428a7a

@ahmedabu98
Copy link
Contributor Author

thanks for the catch @Abacn. would it be good enough to exclude Hive tests based on Java version? similar to the gradle logic in the same PR you linked: https://github.com/apache/beam/pull/32190/files#diff-63a43ac5cc26afc0e5cca037d586671d65fad2b34c2205e88cd46d8531910509R100-R109

@ahmedabu98
Copy link
Contributor Author

Tests in the updated PR (#32232) actually passed, so i went ahead and merged it

@ahmedabu98 ahmedabu98 closed this Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants