Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test: Add unit tests to validate forTable calls setAll with table properties #3902

Closed
wants to merge 4 commits into from
Closed

Test: Add unit tests to validate forTable calls setAll with table properties #3902

wants to merge 4 commits into from

Conversation

hililiwei
Copy link
Contributor

@hililiwei hililiwei commented Jan 14, 2022 •
edited
Loading

This PR is base on #3810 (comment)

Tests whether Parquet\avro following table properties work as expected:

Property Default Description
write.parquet.row-group-size-bytes 134217728 (128 MB) Parquet row group size
write.parquet.page-size-bytes 1048576 (1 MB) Parquet page size
write.parquet.dict-size-bytes 2097152 (2 MB) Parquet dictionary page size
write.parquet.compression-codec gzip Parquet compression codec
write.avro.compression-codec gzip Avro compression codec

@github-actions github-actions bot added the flink label Jan 14, 2022
rdblue
rdblue requested changes Jan 17, 2022
View reviewed changes
Copy link
Contributor

@rdblue rdblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should test that the table settings are being passed to Parquet correctly, rather than trying to make assertions about Parquet files.

@hililiwei hililiwei force-pushed the TestParquetProperties branch from c6a36e1 to 7759dec Compare January 18, 2022 14:56
@hililiwei hililiwei changed the title Parquet: unit test for parquet table properties Test: unit test for parquet\avro table properties Jan 18, 2022
@hililiwei
Copy link
Contributor Author

I think this should test that the table settings are being passed to Parquet correctly, rather than trying to make assertions about Parquet files.

ok, make sense. Rewritten the unit tests. Could you please take a look? thx. @rdblue

@hililiwei hililiwei requested a review from rdblue January 18, 2022 15:01
@hililiwei hililiwei force-pushed the TestParquetProperties branch from 7759dec to 9d32521 Compare January 18, 2022 16:04
@rdblue rdblue changed the title Test: unit test for parquet\avro table properties Test: Add unit tests to validate forTable calls setAll with table properties Jan 18, 2022
@hililiwei hililiwei requested a review from rdblue January 19, 2022 05:57
@hililiwei hililiwei mentioned this pull request Jan 19, 2022
Merged
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions bot added the stale label Aug 7, 2024
@github-actions GitHub Actions
Copy link

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@github-actions github-actions bot closed this Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rdblue rdblue Awaiting requested review from rdblue

Assignees
No one assigned
Labels
core flink parquet stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hililiwei @rdblue