Skip to content

[SPARK-33938][SQL][3.1] Optimize Like Any/All by LikeSimplification #31063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[SPARK-33938][SQL][3.1] Optimize Like Any/All by LikeSimplification #31063

wants to merge 1 commit into from

Conversation

beliefer
Copy link
Contributor

@beliefer beliefer commented Jan 6, 2021

What changes were proposed in this pull request?

We should optimize Like Any/All by LikeSimplification to improve performance.

Why are the changes needed?

Optimize Like Any/All

Does this PR introduce any user-facing change?

'No'.

How was this patch tested?

Jenkins test.

@beliefer
Copy link
Contributor Author

beliefer commented Jan 6, 2021

@cloud-fan

@SparkQA
Copy link

SparkQA commented Jan 6, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38322/

@SparkQA
Copy link

SparkQA commented Jan 6, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38322/

@github-actions github-actions bot added the SQL label Jan 6, 2021
@HyukjinKwon HyukjinKwon changed the title [SPARK-33938][SQL][backport][3.1] Optimize Like Any/All by LikeSimplification [SPARK-33938][SQL][3.1] Optimize Like Any/All by LikeSimplification Jan 7, 2021
@cloud-fan
Copy link
Contributor

retest this please

@SparkQA
Copy link

SparkQA commented Jan 7, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38376/

@SparkQA
Copy link

SparkQA commented Jan 7, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38376/

@SparkQA
Copy link

SparkQA commented Jan 7, 2021

Test build #133788 has finished for PR 31063 at commit e476610.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • sealed abstract class LikeAllBase extends MultiLikeBase
  • sealed abstract class LikeAnyBase extends MultiLikeBase

@cloud-fan
Copy link
Contributor

thanks, merging to 3.1!

cloud-fan pushed a commit that referenced this pull request Jan 8, 2021
@beliefer @cloud-fan
[SPARK-33938][SQL][3.1] Optimize Like Any/All by LikeSimplification
5685a05 
### What changes were proposed in this pull request?
We should optimize Like Any/All by LikeSimplification to improve performance.

### Why are the changes needed?
Optimize Like Any/All

### Does this PR introduce _any_ user-facing change?
'No'.

### How was this patch tested?
Jenkins test.

Closes #31063 from beliefer/SPARK-33938_backport-3.1.

Authored-by: gengjiaan <gengjiaan@360.cn>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
@cloud-fan cloud-fan closed this Jan 8, 2021
@beliefer
Copy link
Contributor Author

beliefer commented Jan 8, 2021

@cloud-fan Thanks for your work

@dongjoon-hyun
Copy link
Member

Although this is an improvement, I'm okay with this backport.

cc @HyukjinKwon .

@HyukjinKwon
Copy link
Member

I am okay too. If it's pretty major and safe, I am okay with porting back.

@cloud-fan
Copy link
Contributor

It's a 3.1 perf regression: #30975 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cloud-fan cloud-fan cloud-fan approved these changes

Assignees
No one assigned
Labels
SQL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@beliefer @SparkQA @cloud-fan @dongjoon-hyun @HyukjinKwon