Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BugFix] Fix full outer join rewrite and nested mv refresh bug for main(#34071) #34299

Merged

Conversation

ABingHuang
Copy link
Contributor

@ABingHuang ABingHuang commented Nov 3, 2023

For rewrite:

  1. fix join derivibility rewrite bug for multi joins query
  2. fix equivalence class construction bugs to exclude outer join on predicate, which should be rewritten by ReplaceColumnRefRewriter
  3. optimize predicate normalization to process column name's case
  4. set mv plan timeout to new_planner_optimize_timeout

For refresh:

  1. fix refresh on nested mv bug to process date format like %Y%m%d
  2. set strict mode to false for mv refresh

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.2
    • 3.1
    • 3.0
    • 2.5

}
return Range.all();
}

/**
* Return the updated partition key ranges of the specific table.
*
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are my suggestions upon reviewing your code:

  1. Error Handling: There's no handling for potential error(s) when parsing the date string, which might throw DateTimeParseException. You should handle this exception in order to prevent it from propagating upwards.

  2. Code Duplication: There is some repetitive code occurring in both convertToDateRange and convertToVarcharRange. This especially holds true for the conditionals checking whether the range has upper and lower bound (both existing independently). To address this, consider refactoring these sections into their own methods. You can create helper functions to carry out these tasks.

  3. Type Checks: The instance type checks with instanceof DateLiteral or instanceof StringLiteral can potentially become problematic if more types are to be supported later on. Consider using a strategy pattern or similar approach for dealing with different types.

  4. Comments: While you have done well to comment your methods, you could improve readability by also describing what the function does in general, not just talking about specific implementation details. These comments will be beneficial for people reading your code, especially people new to the project.

  5. Magic Numbers: The use of .getKeys().get(0) suggests there may be magic numbers being used in your code. This might make your code less understandable and difficult to maintain. If the 0 index has a special significance, either comment it well or factor out a constant with an explanatory name.

  6. Safe Unwrapping: When unpacking values from complex objects like from.lowerEndpoint().getKeys().get(0), there's an assumption that every step retrieves a non-null value. Consider including null-checks or use Optional types to avoid NullPointerExceptions.

  7. Method Naming: The method names such as convertToDateRange and convertToVarcharRange are good and descriptive. Good job on this!

Overall your code seems well-structured, other than these few areas that could be enhanced.

@ABingHuang ABingHuang enabled auto-merge (squash) November 3, 2023 08:08
@ABingHuang ABingHuang force-pushed the fix_full_outer_join_bug_for_main branch 2 times, most recently from 75ab92a to 1fbe6eb Compare November 9, 2023 11:57
@ABingHuang ABingHuang force-pushed the fix_full_outer_join_bug_for_main branch 2 times, most recently from 92e6129 to 6921179 Compare November 14, 2023 03:08
ABingHuang and others added 3 commits November 14, 2023 15:51
…ocks#34071)

For rewrite:
1. fix join derivibility rewrite bug for multi joins query
2. fix equivalence class construction bugs to exclude outer join on predicate,
which should be rewritten by ReplaceColumnRefRewriter
3. optimize predicate normalization to process column name's case
4. set mv plan timeout to new_planner_optimize_timeout

For refresh:
1. fix refresh on nested mv bug to process date format like %Y%m%d
2. set strict mode to false for mv refresh

Signed-off-by: ABingHuang <codekhuang@163.com>
Signed-off-by: ABingHuang <codekhuang@163.com>
Signed-off-by: ABingHuang <codekhuang@163.com>
@ABingHuang ABingHuang force-pushed the fix_full_outer_join_bug_for_main branch from 6921179 to df381aa Compare November 14, 2023 07:53
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 14 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

warning The version of Java (11.0.21) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

Copy link

[BE Incremental Coverage Report]

pass : 0 / 0 (0%)

Copy link

[FE Incremental Coverage Report]

pass : 148 / 153 (96.73%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/scheduler/PartitionBasedMvRefreshProcessor.java 22 25 88.00% [1213, 1214, 1261]
🔵 com/starrocks/sql/optimizer/rule/transformation/materialization/MvUtils.java 25 26 96.15% [1254]
🔵 com/starrocks/sql/optimizer/rule/transformation/materialization/MaterializedViewRewriter.java 89 90 98.89% [407]
🔵 com/starrocks/sql/optimizer/rewrite/scalar/MvNormalizePredicateRule.java 4 4 100.00% []
🔵 com/starrocks/sql/optimizer/MvPlanContextBuilder.java 4 4 100.00% []
🔵 com/starrocks/sql/optimizer/rule/mv/JoinDeriveContext.java 4 4 100.00% []

@ABingHuang ABingHuang merged commit 28ca296 into StarRocks:main Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants