Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KYUUBI #1018] Set spark.sql.execution.topKSortFallbackThreshold to 10000 #1054

Closed
wants to merge 1 commit into from

Conversation

byyue
Copy link
Contributor

@byyue byyue commented Sep 8, 2021

Why are the changes needed?

To avoid performance issues in the topK scenario and close #1018 .

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

@codecov-commenter
Copy link

Codecov Report

Merging #1054 (5ed0ae2) into master (6509a6b) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 5ed0ae2 differs from pull request most recent head 21de1fc. Consider uploading reports for the commit 21de1fc to get more accurate results
Impacted file tree graph

@@            Coverage Diff            @@
##             master    #1054   +/-   ##
=========================================
  Coverage     78.70%   78.70%           
  Complexity       61       61           
=========================================
  Files           169      169           
  Lines          6249     6250    +1     
  Branches        725      725           
=========================================
+ Hits           4918     4919    +1     
  Misses          907      907           
  Partials        424      424           
Impacted Files Coverage Δ
...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala 71.83% <100.00%> (+0.40%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6509a6b...21de1fc. Read the comment docs.

@yaooqinn yaooqinn added this to the v1.4.0 milestone Sep 8, 2021
@yaooqinn yaooqinn closed this in 033784c Sep 9, 2021
@yaooqinn
Copy link
Member

yaooqinn commented Sep 9, 2021

thanks, merged to master

@byyue byyue deleted the feature-1018 branch September 9, 2021 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Set spark.sql.execution.topKSortFallbackThreshold to a reasonable value
4 participants