Skip to content

[SPARK-13306][SQL] Addendum to uncorrelated scalar subquery #11285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

rxin
Copy link
Contributor

@rxin rxin commented Feb 21, 2016

What changes were proposed in this pull request?

This pull request fixes some minor issues (documentation, test flakiness, test organization) with #11190, which was merged earlier tonight.

How was the this patch tested?

unit tests.

@@ -272,14 +272,3 @@ case class Literal protected (value: Any, dataType: DataType)
case _ => value.toString
}
}

// TODO: Specialize
case class MutableLiteral(var value: Any, dataType: DataType, nullable: Boolean = true)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was unused. just deleting it.

@rxin
Copy link
Contributor Author

rxin commented Feb 21, 2016

cc @davies
can you explain what the following test case is testing?

     assertResult(Array(Row(1))) {
       sql("with t2 as (select 1 as b, 2 as c) " +
         "select a from (select 1 as a union all select 2 as a) t " +
         "where a = (select max(b) from t2) ").collect()
     }

@rxin
Copy link
Contributor Author

rxin commented Feb 21, 2016

Also explain why wholestagecodegen needs to call waitforsubqueries explicitly.

@davies
Copy link
Contributor

davies commented Feb 21, 2016

For those operators that is generated in WholeStageCodegen, the execute() of them are not called, but produce() will be called, we should make sure that all the subqueries all executed before generating the code in doProduce() and doConsume()

@davies
Copy link
Contributor

davies commented Feb 21, 2016

That case is used to test CTE and aggregation inside subquery.

@SparkQA
Copy link

SparkQA commented Feb 21, 2016

Test build #51614 has finished for PR 11285 at commit de512f6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Feb 21, 2016

Test build #51611 has finished for PR 11285 at commit c9bd987.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PlanSubqueries(sqlContext: SQLContext) extends Rule[SparkPlan]

@rxin
Copy link
Contributor Author

rxin commented Feb 21, 2016

Merging this in master.

@asfgit asfgit closed this in af441dd Feb 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants