[SPARK-9297][SQL] Add covar_pop and covar_samp

[viirya]

JIRA: https://issues.apache.org/jira/browse/SPARK-9297

Add two aggregation functions: covar_pop and covar_samp.

[SparkQA]

Test build #46849 has finished for PR 10029 at commit 5a419f6.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * abstract class Covariance(\n * case class CovSample(\n * case class CovPopulation(\n

[SparkQA]

Test build #46851 has finished for PR 10029 at commit f0c1838.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * abstract class Covariance(\n * case class CovSample(\n * case class CovPopulation(\n

[viirya]

retest this please.

[SparkQA]

Test build #46853 has finished for PR 10029 at commit f0c1838.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * abstract class Covariance(\n * case class CovSample(\n * case class CovPopulation(\n

[davies]

2.0.0

[SparkQA]

Test build #49128 has finished for PR 10029 at commit 45d9f0c.

• This patch fails from timeout after a configured wait of 250m.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

retest this please.

[SparkQA]

Test build #49146 has finished for PR 10029 at commit 45d9f0c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

retest this please.

[SparkQA]

Test build #49150 has finished for PR 10029 at commit 45d9f0c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

@davies please see if the updates are good for you. Thanks.

[viirya]

ping @davies

[davies]

Minor: Should we check Ck is nan or not? since count > 0

[viirya]

We check if cov is nan or not. If Ck is nan, cov will be nan too.

[davies]

I meant it make more sense to check Ck directly, because count > 0.

[viirya]

oh. Got it.

[davies]

@viirya One question is that: we are moving to do more codegen staff (whole stage codegen), but the ImperativeAggregate can't support that, not sure will we do another codegen version for this or not. Right now, ImperativeAggregate version faster, because the generated code is still not the best, but we can improve.

[viirya]

@davies Understood. Since the time to come out a better codegen version for this is not sure, can we merge this first in and update it with codegen version later if possibly?

[davies]

@viirya Yeah, we can merge this in first.

[SparkQA]

Test build #49300 has finished for PR 10029 at commit 832db06.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • abstract class Covariance(left: Expression, right: Expression) extends ImperativeAggregate

[davies]

If the result is null, we should combine there two branches.

[SparkQA]

Test build #49309 has finished for PR 10029 at commit 1806ced.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #49305 has finished for PR 10029 at commit 2f643d4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[davies]

LGTM, merging this into master, thanks!