Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix TiFlash overflow #2740

Merged
merged 3 commits into from
Aug 1, 2023
Merged

Fix TiFlash overflow #2740

merged 3 commits into from
Aug 1, 2023

Conversation

shiyuhang0
Copy link
Member

@shiyuhang0 shiyuhang0 commented Jul 31, 2023 •
edited
Loading

What problem does this PR solve?

Reproduce:

create and insert

CREATE TABLE test.t(`a` bigint(20) UNSIGNED NOT NULL)
insert into t values (16717361816800086255)

query from tispark with tiflash returns an error result（negative number）

 spark.conf
        .set("spark.tispark.isolation_read_engines","tiflash")
spark.sql("select * from tidb_catalog.test.t").show()

What is changed and how it works?

TiSpark requests TiFlash using TypeCHBlock encode, but forgets to handle the UNSIGNED scenes when decoded.

this pr shows how to decode it corretly

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to update the tidb-ansible repository
  • Need to be included in the release note

@ti-chi-bot ti-chi-bot bot requested a review from zhangyangyu July 31, 2023 14:05
@ti-chi-bot ti-chi-bot bot added the size/M label Jul 31, 2023
@shiyuhang0
Copy link
Member Author

/run-all-tests

@shiyuhang0
Copy link
Member Author

/run-all-tests

@shiyuhang0
Copy link
Member Author

/run-all-tests

1 similar comment
@shiyuhang0
Copy link
Member Author

/run-all-tests

@shiyuhang0 shiyuhang0 changed the title fix tiflash overflow Fix TiFlash overflow Aug 1, 2023
@shiyuhang0 shiyuhang0 added needs-cherry-pick-release-2.4 PR which needs to be cherry-picked to release-2.4 needs-cherry-pick-release-2.5 PR which needs to be cherry-picked to release-2.5 needs-cherry-pick-release-3.0 PR which needs to be cherry-picked to release-2.5 needs-cherry-pick-release-3.1 PR which needs to be cherry-picked to release-3.1 needs-cherry-pick-release-3.2 labels Aug 1, 2023
@shiyuhang0
Copy link
Member Author

/run-all-tests

@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 1, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xuanyu66, zhangyangyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 1, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-08-01 03:24:55.913210286 +0000 UTC m=+155779.855558814: ☑️ agreed by xuanyu66.
  • 2023-08-01 07:01:14.647476771 +0000 UTC m=+168758.589825299: ☑️ agreed by zhangyangyu.

@ti-chi-bot ti-chi-bot bot merged commit 0446309 into pingcap:master Aug 1, 2023
11 checks passed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-2.5: #2741.

@ti-chi-bot ti-chi-bot mentioned this pull request Aug 1, 2023
Open
ti-chi-bot pushed a commit to ti-chi-bot/tispark that referenced this pull request Aug 1, 2023
@shiyuhang0 @ti-chi-bot
This is an automated cherry-pick of pingcap#2740
b789c7b 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Aug 1, 2023
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-3.0: #2742.

ti-chi-bot pushed a commit to ti-chi-bot/tispark that referenced this pull request Aug 1, 2023
@shiyuhang0 @ti-chi-bot
This is an automated cherry-pick of pingcap#2740
b3387cc 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Aug 1, 2023
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-3.1: #2743.

ti-chi-bot pushed a commit to ti-chi-bot/tispark that referenced this pull request Aug 1, 2023
@shiyuhang0 @ti-chi-bot
This is an automated cherry-pick of pingcap#2740
7df1f85 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-3.2: #2744.

@ti-chi-bot ti-chi-bot mentioned this pull request Aug 1, 2023
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-2.4: #2745.

@ti-chi-bot ti-chi-bot mentioned this pull request Aug 1, 2023
Open
ti-chi-bot pushed a commit to ti-chi-bot/tispark that referenced this pull request Aug 1, 2023
@shiyuhang0 @ti-chi-bot
This is an automated cherry-pick of pingcap#2740
fbb0df4 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
shiyuhang0 added a commit that referenced this pull request Aug 2, 2023
@ti-chi-bot @shiyuhang0
Fix TiFlash overflow (#2740) (#2743)
fe87946 
* This is an automated cherry-pick of #2740

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

* Update TiBlockColumnVector.java

* fmt

---------

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
Co-authored-by: shi yuhang <52435083+shiyuhang0@users.noreply.github.com>
Co-authored-by: shiyuhang <1136742008@qq.com>
shiyuhang0 pushed a commit that referenced this pull request Aug 8, 2023
@ti-chi-bot
Fix TiFlash overflow (#2740) (#2744)
95c0d90
shiyuhang0 pushed a commit that referenced this pull request Aug 8, 2023
@ti-chi-bot
Fix TiFlash overflow (#2740) (#2742)
75fd1c0
@zhangyangyu zhangyangyu deleted the syh/tiflash_overflow branch January 31, 2024 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhangyangyu zhangyangyu zhangyangyu approved these changes

@xuanyu66 xuanyu66 xuanyu66 approved these changes

Assignees
No one assigned
Labels
approved lgtm needs-cherry-pick-release-2.4 PR which needs to be cherry-picked to release-2.4 needs-cherry-pick-release-2.5 PR which needs to be cherry-picked to release-2.5 needs-cherry-pick-release-3.0 PR which needs to be cherry-picked to release-2.5 needs-cherry-pick-release-3.1 PR which needs to be cherry-picked to release-3.1 needs-cherry-pick-release-3.2 size/M
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@shiyuhang0 @ti-chi-bot @zhangyangyu @xuanyu66