Skip to content

[SPARK-18214][SQL] Simplify RuntimeReplaceable type coercion #15723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from

Conversation

rxin
Copy link
Contributor

@rxin rxin commented Nov 2, 2016

What changes were proposed in this pull request?

RuntimeReplaceable is used to create aliases for expressions, but the way it deals with type coercion is pretty weird (each expression is responsible for how to handle type coercion, which does not obey the normal implicit type cast rules).

This patch simplifies its handling by allowing the analyzer to traverse into the actual expression of a RuntimeReplaceable.

How was this patch tested?

  • Correctness should be guaranteed by existing unit tests already
  • Removed SQLCompatibilityFunctionSuite and moved it sql-compatibility-functions.sql
  • Added a new test case in sql-compatibility-functions.sql for verifying explain behavior.

@marmbrus
Copy link
Contributor

marmbrus commented Nov 2, 2016

LGTM

@SparkQA
Copy link

SparkQA commented Nov 2, 2016

Test build #67935 has finished for PR 15723 at commit 8cdf56c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait RuntimeReplaceable extends UnaryExpression with Unevaluable
    • case class IfNull(left: Expression, right: Expression, child: Expression)
    • case class NullIf(left: Expression, right: Expression, child: Expression)
    • case class Nvl(left: Expression, right: Expression, child: Expression) extends RuntimeReplaceable
    • case class Nvl2(expr1: Expression, expr2: Expression, expr3: Expression, child: Expression)

@SparkQA
Copy link

SparkQA commented Nov 2, 2016

Test build #67936 has finished for PR 15723 at commit 75e7839.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 2, 2016

Test build #3393 has finished for PR 15723 at commit a5c0b16.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 2, 2016

Test build #67953 has finished for PR 15723 at commit a5c0b16.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 2, 2016

Test build #67961 has finished for PR 15723 at commit 5face4a.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 2, 2016

Test build #67963 has finished for PR 15723 at commit 01cfa00.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor Author

rxin commented Nov 2, 2016

Merging in master/branch-2.1.

@asfgit asfgit closed this in fd90541 Nov 2, 2016
asfgit pushed a commit that referenced this pull request Nov 2, 2016
## What changes were proposed in this pull request?
RuntimeReplaceable is used to create aliases for expressions, but the way it deals with type coercion is pretty weird (each expression is responsible for how to handle type coercion, which does not obey the normal implicit type cast rules).

This patch simplifies its handling by allowing the analyzer to traverse into the actual expression of a RuntimeReplaceable.

## How was this patch tested?
- Correctness should be guaranteed by existing unit tests already
- Removed SQLCompatibilityFunctionSuite and moved it sql-compatibility-functions.sql
- Added a new test case in sql-compatibility-functions.sql for verifying explain behavior.

Author: Reynold Xin <rxin@databricks.com>

Closes #15723 from rxin/SPARK-18214.

(cherry picked from commit fd90541)
Signed-off-by: Reynold Xin <rxin@databricks.com>
uzadude pushed a commit to uzadude/spark that referenced this pull request Jan 27, 2017
## What changes were proposed in this pull request?
RuntimeReplaceable is used to create aliases for expressions, but the way it deals with type coercion is pretty weird (each expression is responsible for how to handle type coercion, which does not obey the normal implicit type cast rules).

This patch simplifies its handling by allowing the analyzer to traverse into the actual expression of a RuntimeReplaceable.

## How was this patch tested?
- Correctness should be guaranteed by existing unit tests already
- Removed SQLCompatibilityFunctionSuite and moved it sql-compatibility-functions.sql
- Added a new test case in sql-compatibility-functions.sql for verifying explain behavior.

Author: Reynold Xin <rxin@databricks.com>

Closes apache#15723 from rxin/SPARK-18214.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants