Skip to content

[Bug] Compaction leads to schema change error #9032

Closed
@liutang123

Description

Search before asking

  • I had searched in the issues and found no similar issues.

Version

0.14

What's Wrong?

After #6365, doris may do comapction when a tablet in altering.

base tablet:[0-62][63-64][65-65][66-66][67-67]
new tablet:[50-65][66-66][67-67]
begin compaction in new tablet: [50-65][66-66]
begin schema change converting: begin to remove all data from new tablet to prevent rewrite.
new tablet: [50-65][66-66][67-67] will move to TimestampedVersionTracker._stale_version_path_map
compaction end in new tablet: [50-66] will be inserted into new tablet.
begin convert base tablet to new tablet:
insert [0-62] to new tablet and new tablet will be[0-62][50-66]
insert [63-64] will fail.
version already exist, version revert occurred. tablet=XXXXX, version='63-64
[65-65] and [66-66] will fail too.
version already exist, version revert occurred. tablet=XXXXX, version='66-66
insert [67-67] to new tablet, and new tabelt will be: [0-62][50-66][67]
Because the removed rowset can be query now, check_version_integrity([0-67]) will success and the schema cahnge will success.

I0412 21:42:58.683157 60051 schema_change.cpp:1953] finish converting rowsets for new_tablet from base_tablet. base_tablet=XXXXX, new_tablet=XXXXX
I0412 21:42:58.700071 60051 engine_alter_tablet_task.cpp:52] success to create new alter tablet. res=0 base_tablet_id=XXXX, base_schema_hashXXXX, new_tablet_id=XXXX, new_schema_hash=XXXX

What You Expected?

  1. compaction should not add output rowset to tablet if the origin rowsets not in tablet's _rs_version_map.
  2. schema change should check new tablet's _rs_version_map
  3. tablet report should check if a tablet has intersected versions.

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions