[feat] join reorder by table owner #697
Conversation
Summary of ChangesHello @tyrone-yu, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant enhancement to the query optimizer by implementing a join reordering strategy that is sensitive to table ownership. By incorporating a Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a join reordering optimization to prioritize local joins over cross-party joins. This is a valuable performance enhancement for distributed queries. The implementation adds PartyCode to schemas to track data locality and uses a greedy algorithm with a cost model that favors local joins. The changes are well-structured and include relevant tests.
I've found a critical issue in the join group extraction logic that could lead to incorrect query results by reordering non-inner joins, and a medium-severity performance issue in a utility function. Addressing these will make the implementation more robust and efficient.
| Outputs: [] | ||
| } | ||
| `, testConf{groupThreshold: 0, batched: true, revealGroupCount: false}}, | ||
| {`select count(*) from alice.tbl_1 as t1 join alice.tbl_2 as t2 join bob.tbl_1 as t3 join bob.tbl_2 as t4 where t1.plain_int_0 = t2.plain_int_0 and t2.plain_int_0 = t3.plain_int_0 and t3.plain_int_0 = t4.plain_int_0`, `digraph G { |
There was a problem hiding this comment.
例子里面是不是 t2 t3调整个位置验证下？
| // results in a join group {a, b, LeftJoin(c, d)}. | ||
| func extractJoinGroup(p LogicalPlan) (group []LogicalPlan, eqEdges []*expression.ScalarFunction, otherConds []expression.Expression) { | ||
| join, isJoin := p.(*LogicalJoin) | ||
| if !isJoin || join.JoinType != InnerJoin { |
There was a problem hiding this comment.
请教下：这里会导致策略只对inner join 生效吗？
There was a problem hiding this comment.
目前就只支持 inner join，left/right join reorder 现在好像不能保证结果正确，先不支持
| eqEdges: eqEdges, | ||
| } | ||
| p, err = groupSolver.solve(curJoinGroup) | ||
| // TODO: @xiaoyuan support Dp solver |
There was a problem hiding this comment.
请教一下，这里的dp solver本来是干嘛的呀
There was a problem hiding this comment.
这里 join reorder 就是通过计算 join 的 cost 去寻找估值最低的顺序，相当于现在使用的是暴露解法，还可以通过 dp 的思路优化耗时，不过在 join 的表数据较大的时候才会产生效果，现在用不上
There was a problem hiding this comment.
tyrone-yu
force-pushed
the
yyq/join-reorder
branch
from
d2cab2c to
d6ab52e
Compare
No description provided.