You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In apache/spark#33904, I faced a performance issue with the topK scenario where K is very large which can be avoided by setting spark.sql.execution.topKSortFallbackThreshold less than the K here.
For Kyuubi users who run SQL only, improper topK SQLs will hold engines for quite a long time and they don't know why.
With a proper value for spark.sql.execution.topKSortFallbackThreshold is more suitable for Kyuubi's cases
I'd suggest this value be set to 10000
2. Motivation
3. Describe the solution
4. Additional context
The text was updated successfully, but these errors were encountered:
1. Describe the feature
In apache/spark#33904, I faced a performance issue with the topK scenario where K is very large which can be avoided by setting spark.sql.execution.topKSortFallbackThreshold less than the K here.
For Kyuubi users who run SQL only, improper topK SQLs will hold engines for quite a long time and they don't know why.
With a proper value for spark.sql.execution.topKSortFallbackThreshold is more suitable for Kyuubi's cases
I'd suggest this value be set to 10000
2. Motivation
3. Describe the solution
4. Additional context
The text was updated successfully, but these errors were encountered: