chore: refine copy into tests #17413
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The tests should not assert the exact byte size of data outputs, as this value may change across different versions of `databend-query` due to meta data of data block contains the version the `databend-query`, which is variant lengthed. This change replaces byte size checks with content validation, ensuring test stability while maintaining functional correctness.
github-actions
bot
added
the
pr-chore
05_0001_set_var.sh
18_0002_ownership_cover
18_0007_privilege_access
20_0015_compact_hook_stas_issue_13947
11 tasks
SkyFan2002
approved these changes
Feb 6, 2025
zhyass
approved these changes
Feb 6, 2025
dantengsky
added a commit
to dantengsky/fuse-query
that referenced
this pull request
Feb 6, 2025
* test(stage): decouple `COPY INTO` tests from output data byte size The tests should not assert the exact byte size of data outputs, as this value may change across different versions of `databend-query` due to meta data of data block contains the version the `databend-query`, which is variant lengthed. This change replaces byte size checks with content validation, ensuring test stability while maintaining functional correctness. * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size 05_0001_set_var.sh * test(stage): decouple `COPY INTO` tests from output data byte size 18_0002_ownership_cover * decouple `COPY INTO` tests from output data byte size 18_0007_privilege_access * decouple `COPY INTO` tests from output data byte size 20_0015_compact_hook_stas_issue_13947 * decouple `COPY INTO` tests from output data byte size * revert scripts/ci/deploy/config/databend-query-node-1.toml
11 tasks
dantengsky
added a commit
that referenced
this pull request
May 23, 2025
) * refactor: rename components of the `replace into` impl (#17355) refactor: rename components of `replace into` Initially, `REPLACE INTO` was designed as a simplified implementation of `MERGE INTO`, with the expectation that it would gradually evolve into a full-fledged implementation of `MERGE INTO`, as a result, many `MERGE INTO` related terms exist in the components of `REPLACE INTO` impl. However, `MERGE INTO` now has its own independent impl, the remaining `MERGE INTO` related terms in the impl of `REPLACE INTO` can be misleading. Therefore, in this PR, we have renamed all `MERGE INTO` related terms in the implof `REPLACE INTO` to avoid confusion. * feat: introduce block meta cache (#17360) * feat: introduce BlockMetaCache * tweak sys tbl `system.cache` and configs * cargo fmt * update ut gloden file * fix: set cache cap of `memory_cache_block_meta` * tweak cargo audit.toml * chore: refine copy into tests (#17413) * test(stage): decouple `COPY INTO` tests from output data byte size The tests should not assert the exact byte size of data outputs, as this value may change across different versions of `databend-query` due to meta data of data block contains the version the `databend-query`, which is variant lengthed. This change replaces byte size checks with content validation, ensuring test stability while maintaining functional correctness. * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size * test(stage): decouple `COPY INTO` tests from output data byte size 05_0001_set_var.sh * test(stage): decouple `COPY INTO` tests from output data byte size 18_0002_ownership_cover * decouple `COPY INTO` tests from output data byte size 18_0007_privilege_access * decouple `COPY INTO` tests from output data byte size 20_0015_compact_hook_stas_issue_13947 * decouple `COPY INTO` tests from output data byte size * revert scripts/ci/deploy/config/databend-query-node-1.toml * tweak test case parquet_missing_uuid ignore result-set of `copy into` when preparing data. * tweak 00_0015_unload_output, 00_0012_stage_with_connection * refactor: refine `replace into` by caching individual BlockMeta (#17368) * remove dead code * update crossbeam-channel to 0.5.13 Title: crossbeam-channel: double free on Drop Date: 2025-04-08 ID: RUSTSEC-2025-0024 URL: https://rustsec.org/advisories/RUSTSEC-2025-0024 * update audit.toml * tweak logic test 05_0001_set_var bendsql used in CI has been upgraded which uses a different format to display timestamps. * chore: improve license validation log message (#17967) * chore: improve license validation log message Change license validation error log to ERROR level. Several enterprise features don't affect functional correctness but impact performance significantly. Users have experienced unexplained performance degradation due to expired licenses. This change helps users identify such issues faster. * improve the license error message * tweak log msg --------- Co-authored-by: BohuTANG <overred.shuttler@gmail.com> --------- Co-authored-by: BohuTANG <overred.shuttler@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
Some test cases rely on the exact size of the parquet files generated by the
COPY INTO $locationstatement, which is flaky. The size of the generated parquet files can vary across different versions of databend-query, as the parquet file metadata includes the version of databend-query that created the file.Refer to: #17220
Tests
Type of change
This change is