Skip to content

Remove unused columns when rebuilding projection on merge.#93233

Merged
KochetovNicolai merged 1 commit intomasterfrom
remove-unused-columns-when-rebuild-projection-on-merge
Jan 2, 2026
Merged

Remove unused columns when rebuilding projection on merge.#93233
KochetovNicolai merged 1 commit intomasterfrom
remove-unused-columns-when-rebuild-projection-on-merge

Conversation

@KochetovNicolai
Copy link
Member

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Remove unused columns when the projection is rebuilt during the merge. It reduces memory usage and creates fewer temporary parts.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@clickhouse-gh
Copy link
Contributor

clickhouse-gh bot commented Dec 30, 2025

Workflow [PR], commit [928559d]

Summary:

job_name test_name status info comment
Integration tests (amd_asan, flaky) failure
test_projection_rebuild_with_required_columns/test.py::test_projection_rebuild_uses_only_required_columns[2-3] FAIL cidb IGNORED
test_projection_rebuild_with_required_columns/test.py::test_projection_rebuild_uses_only_required_columns[3-3] FAIL cidb IGNORED

@clickhouse-gh clickhouse-gh bot added the pr-bugfix Pull request with bugfix, not backported by default label Dec 30, 2025
@amosbird
Copy link
Collaborator

I've also optimized this part (see https://github.com/ClickHouse/ClickHouse/pull/93114/changes#diff-6c14341ca43dbbaef007e622cb9553ef92f4f32d3f0fd8c150ab7567678df545R984) by leveraging the fact that generated blocks always have monotonically increasing part offsets (both in merges and mutations). This allows us to pass only the starting offset to compute and construct the offset column efficiently.

@KochetovNicolai
Copy link
Member Author

@amosbird thank you!
I would like to keep my PR as well, because it's backportable.

@Avogar Avogar self-assigned this Dec 30, 2025
@KochetovNicolai KochetovNicolai added this pull request to the merge queue Jan 2, 2026
Merged via the queue into master with commit 6c780e3 Jan 2, 2026
129 of 131 checks passed
@KochetovNicolai KochetovNicolai deleted the remove-unused-columns-when-rebuild-projection-on-merge branch January 2, 2026 10:49
@KochetovNicolai KochetovNicolai added the pr-must-backport Pull request should be backported intentionally. Use this label with great care! label Jan 2, 2026
@robot-ch-test-poll4 robot-ch-test-poll4 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jan 2, 2026
@robot-ch-test-poll1 robot-ch-test-poll1 added the pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR label Jan 2, 2026
robot-ch-test-poll2 added a commit that referenced this pull request Jan 2, 2026
Cherry pick #93233 to 25.8: Remove unused columns when rebuilding projection on merge.
robot-clickhouse added a commit that referenced this pull request Jan 2, 2026
robot-ch-test-poll2 added a commit that referenced this pull request Jan 2, 2026
Cherry pick #93233 to 25.10: Remove unused columns when rebuilding projection on merge.
robot-clickhouse added a commit that referenced this pull request Jan 2, 2026
robot-ch-test-poll2 added a commit that referenced this pull request Jan 2, 2026
Cherry pick #93233 to 25.11: Remove unused columns when rebuilding projection on merge.
robot-clickhouse added a commit that referenced this pull request Jan 2, 2026
robot-ch-test-poll2 added a commit that referenced this pull request Jan 2, 2026
Cherry pick #93233 to 25.12: Remove unused columns when rebuilding projection on merge.
robot-clickhouse added a commit that referenced this pull request Jan 2, 2026
clickhouse-gh bot added a commit that referenced this pull request Jan 2, 2026
Backport #93233 to 25.12: Remove unused columns when rebuilding projection on merge.
clickhouse-gh bot added a commit that referenced this pull request Jan 2, 2026
Backport #93233 to 25.11: Remove unused columns when rebuilding projection on merge.
clickhouse-gh bot added a commit that referenced this pull request Jan 2, 2026
Backport #93233 to 25.10: Remove unused columns when rebuilding projection on merge.
clickhouse-gh bot added a commit that referenced this pull request Jan 2, 2026
Backport #93233 to 25.8: Remove unused columns when rebuilding projection on merge.
<clickhouse>
<profiles>
<default>
<min_insert_block_size_bytes>1048576</min_insert_block_size_bytes>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this test - what does it do?
As it fails every time, I will remove it.

robot-clickhouse-ci-1 added a commit that referenced this pull request Jan 23, 2026
Cherry pick #93233 to 25.3: Remove unused columns when rebuilding projection on merge.
robot-clickhouse added a commit that referenced this pull request Jan 23, 2026
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Jan 23, 2026
@robot-clickhouse robot-clickhouse added pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore and removed pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore labels Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-bugfix Pull request with bugfix, not backported by default pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants