[SPARK-13366] Support Cartesian join for Datasets

[xguo27]

No description provided.

[marmbrus]

ok to test

[marmbrus]

You don't actually need to use a true literal here, just construct a join without a condition.

[xguo27]

Thanks for your feedback @marmbrus . The only join API in Dataset I can find is:

https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L644

which expects a Column. Do you mean to add some other method like the one in Dataframe:

https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala#L383-L385

If so, I'm wondering whether we need to refactor out the code that handles encoder?

[marmbrus]

I guess I would just make a protected version of joinWith above where the condition is an Option and have all the other methods delegate to that.

[SparkQA]

Test build #51450 has finished for PR 11244 at commit 27a58df.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[xguo27]

Thanks @marmbrus ! I have updated the change following your suggestion.

[SparkQA]

Test build #51609 has finished for PR 11244 at commit c79ec51.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[marmbrus]

@liancheng is this going to be done by your unification? Should we merge or hold off?

[rxin]

Thanks for the pull request. I'm going through a list of pull requests to cut them down since the sheer number is breaking some of the tooling we have. Due to lack of activity on this pull request, I'm going to push a commit to close it. Feel free to reopen it or create a new one. We can also continue the discussion on the JIRA ticket.