Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not recompress segmentwise when default order by is empty #7789

Conversation

natalya-aksman
Copy link
Contributor

@natalya-aksman natalya-aksman commented Mar 4, 2025

When we have compression where default order by is set to "", we cannot do meaningful segmentwise recompression. Therefore we will return error when segmentwise recompression is requested directly and will do full recompression when general compression is requested.

Fixes: #7748

@CLAassistant
Copy link

CLAassistant commented Mar 4, 2025

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link

codecov bot commented Mar 4, 2025

Codecov Report

Attention: Patch coverage is 78.57143% with 3 lines in your changes missing coverage. Please review.

Project coverage is 81.86%. Comparing base (59f50f2) to head (1eba81e).
Report is 803 commits behind head on main.

Files with missing lines Patch % Lines
tsl/src/compression/recompress.c 50.00% 0 Missing and 2 partials ⚠️
tsl/src/compression/api.c 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7789      +/-   ##
==========================================
+ Coverage   80.06%   81.86%   +1.80%     
==========================================
  Files         190      247      +57     
  Lines       37181    45442    +8261     
  Branches     9450    11364    +1914     
==========================================
+ Hits        29770    37203    +7433     
- Misses       2997     3766     +769     
- Partials     4414     4473      +59     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@svenklemm svenklemm requested a review from antekresic March 4, 2025 16:20
@svenklemm svenklemm added this to the v2.19.0 milestone Mar 4, 2025
@natalya-aksman natalya-aksman force-pushed the crash_segmentwise_recompression_i7748 branch from 9738e7a to 224376b Compare March 4, 2025 19:40
@natalya-aksman natalya-aksman force-pushed the crash_segmentwise_recompression_i7748 branch from 224376b to 1eba81e Compare March 4, 2025 19:51
@natalya-aksman natalya-aksman merged commit d7c4244 into timescale:main Mar 4, 2025
47 of 48 checks passed
@timescale-automation
Copy link
Member

Automated backport to 2.18.x not done: cherry-pick failed.

Git status

HEAD detached at origin/2.18.x
You are currently cherry-picking commit d7c424441.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   .unreleased/pr_7789
	modified:   tsl/src/compression/create.c
	modified:   tsl/src/compression/recompress.c

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   tsl/src/compression/api.c
	both modified:   tsl/test/expected/recompress_chunk_segmentwise.out
	both modified:   tsl/test/sql/recompress_chunk_segmentwise.sql


Job log

@timescale-automation timescale-automation added the auto-backport-not-done Automated backport of this PR has failed non-retriably (e.g. conflicts) label Mar 4, 2025
philkra added a commit that referenced this pull request Mar 12, 2025
## 2.19.0 (2025-03-12)

This release contains performance improvements and bug fixes since 
the 2.18.2 release. We recommend that you upgrade at the next 
available opportunity.

**Features**
* [#7586](#7586) Vectorized aggregation with grouping by a single text column.
* [#7632](#7632) Optimize recompression for chunks without segmentby
* [#7655](#7655) Support vectorized aggregation on Hypercore TAM
* [#7669](#7669) Add support for merging compressed chunks
* [#7701](#7701) Implement a custom compression algorithm for bool columns. It is experimental and can undergo backwards-incompatible changes. For testing, enable it using timescaledb.enable_bool_compression = on.
* [#7707](#7707) Support ALTER COLUMN SET NOT NULL on compressed chunks
* [#7765](#7765) Allow tsdb as alias for timescaledb in WITH and SET clauses
* [#7786](#7786) Show warning for inefficient compress_chunk_time_interval configuration
* [#7788](#7788) Add callback to mem_guard for background workers
* [#7789](#7789) Do not recompress segmentwise when default order by is empty
* [#7790](#7790) Add configurable Incremental CAgg Refresh Policy

**Bugfixes**
* [#7665](#7665) Block merging of frozen chunks
* [#7673](#7673) Don't abort additional INSERTs when hitting first conflict
* [#7714](#7714) Fixes a wrong result when compressed NULL values were confused with default values. This happened in very special circumstances with alter table added a new column with a default value, an update and compression in a very particular order.
* [#7747](#7747) Block TAM rewrites with incompatible GUC setting
* [#7748](#7748) Crash in the segmentwise recompression
* [#7764](#7764) Fix compression settings handling in Hypercore TAM
* [#7768](#7768) Remove costing index scan of hypertable parent
* [#7799](#7799) Handle DEFAULT table access name in ALTER TABLE

**Thanks**
* @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks
* @kav23alex for reporting a segmentation fault on ALTER TABLE with DEFAULT

Signed-off-by: Philip Krauss <35487337+philkra@users.noreply.github.com>
This was referenced Mar 12, 2025
philkra added a commit that referenced this pull request Mar 18, 2025
## 2.19.0 (2025-03-18)

This release contains performance improvements and bug fixes since the
2.18.2 release. We recommend that you upgrade at the next available
opportunity.

* Improved concurrency of INSERT, UPDATE and DELETE operations on the
columnstore by no longer blocking DML statements during the
recompression of a chunk.
* Improved system performance during Continuous Aggregates refreshes by
breaking them into smaller batches which reduces systems pressure and
minimizes the risk of spilling to disk.
* Faster and more up-to-date results for queries against Continuous
Aggregates by materializing the most recent data first (vs old data
first in prior versions).
* Faster analytical queries with SIMD vectorization of aggregations over
text columns and group by over multiple column
* Enable optimizing chunk size for faster query performance on the
columnstore by adding support for merging columnstore chunks to the
merge_chunk API.

**Deprecation warning**

This is the last minor release supporting PostgreSQL 14. Starting with
the minor version of TimescaleDB only Postgres 15, 16 and 17 will be
supported.

**Downgrading of 2.19.0**

This release introduces custom bool compression, if you enable this
feature via the `enable_bool_compression` and must downgrade to a
previous, please use the [following
script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.19.0-downgrade_new_compression_algorithms.sql)
to convert the columns back to their previous state. TimescaleDB
versions prior to 2.19.0 do not know how to handle this new type.

**Features**
* [#7586](#7586) Vectorized
aggregation with grouping by a single text column.
* [#7632](#7632) Optimize
recompression for chunks without segmentby
* [#7655](#7655) Support
vectorized aggregation on Hypercore TAM
* [#7669](#7669) Add
support for merging compressed chunks
* [#7701](#7701) Implement
a custom compression algorithm for bool columns. It is experimental and
can undergo backwards-incompatible changes. For testing, enable it using
timescaledb.enable_bool_compression = on.
* [#7707](#7707) Support
ALTER COLUMN SET NOT NULL on compressed chunks
* [#7765](#7765) Allow tsdb
as alias for timescaledb in WITH and SET clauses
* [#7786](#7786) Show
warning for inefficient compress_chunk_time_interval configuration
* [#7788](#7788) Add
callback to mem_guard for background workers
* [#7789](#7789) Do not
recompress segmentwise when default order by is empty
* [#7790](#7790) Add
configurable Incremental CAgg Refresh Policy

**Bugfixes**
* [#7665](#7665) Block
merging of frozen chunks
* [#7673](#7673) Don't
abort additional INSERTs when hitting first conflict
* [#7714](#7714) Fixes a
wrong result when compressed NULL values were confused with default
values. This happened in very special circumstances with alter table
added a new column with a default value, an update and compression in a
very particular order.
* [#7747](#7747) Block TAM
rewrites with incompatible GUC setting
* [#7748](#7748) Crash in
the segmentwise recompression
* [#7764](#7764) Fix
compression settings handling in Hypercore TAM
* [#7768](#7768) Remove
costing index scan of hypertable parent
* [#7799](#7799) Handle
DEFAULT table access name in ALTER TABLE

**GUCs**
* `enable_bool_compression`: enable the BOOL compression algorithm,
default: `OFF`
* `enable_exclusive_locking_recompression`: enable exclusive locking
during recompression (legacy mode), default: `OFF`

**Thanks**
* @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT
DO NOTHING on compressed chunks
* @kav23alex for reporting a segmentation fault on ALTER TABLE with
DEFAULT

---------

Signed-off-by: Philip Krauss <35487337+philkra@users.noreply.github.com>
Signed-off-by: Ramon Guiu <ramon@timescale.com>
Co-authored-by: Ramon Guiu <ramon@timescale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport-not-done Automated backport of this PR has failed non-retriably (e.g. conflicts)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Crash in the segmentwise recompression
5 participants