Find minimal col dim when its not provided #1295
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: This diff has two parts. In the first part, we handle the case when the per table base_dim is not given. We use a for loop for find the base_dim that is at least 128, divides the table dim, and is divisible by 4. This helps with the case when the table dim is not divisible by 128, since in that case the current code would just use the table dim. In the second part, we replace `col_wise_shard_dim` with `_find_base_dim(col_wise_shard_dim, columns)`. In cases where `col_wise_shard_dim` is correctly provided, this does nothing. But this allows the user to put in a more flexible `col_wise_shard_dim`. For example, the users can just put in `col_wise_shard_dim = 40` for all tables, and the planner will find the per table base dim that (1) divides the table dim, (2) is divisible by 4 and (3) is larger than or equal to 40. Differential Revision: D47887146 fbshipit-source-id: 1dfb56e6254a0957cbc7ab5a4bbbdca9590b4ba9
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D47887146 |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This diff has two parts.
In the first part, we handle the case when the per table base_dim is not given. We use a for loop for find the base_dim that is at least 128, divides the table dim, and is divisible by 4. This helps with the case when the table dim is not divisible by 128, since in that case the current code would just use the table dim.
In the second part, we replace
col_wise_shard_dimwith_find_base_dim(col_wise_shard_dim, columns). In cases wherecol_wise_shard_dimis correctly provided, this does nothing. But this allows the user to put in a more flexiblecol_wise_shard_dim. For example, the users can just put incol_wise_shard_dim = 40for all tables, and the planner will find the per table base dim that (1) divides the table dim, (2) is divisible by 4 and (3) is larger than or equal to 40.Differential Revision: D47887146