[SPARK-6964][SQL] Support Cancellation in the Thrift Server #6207

+
+      val largeJoin = "SELECT COUNT(*) FROM test_map " +
+        List.fill(10)("join test_map").mkString(" ")
+      val f = future { Thread.sleep(100); statement.cancel(); }


As discussed off-line we should look at doing this without a sleep, but I don't think that needs to block merging.

HiveStatement does not support setQueryTimeout.

I think that this is the cause of timeouts / flakiness in this suite: on a slow machine, it's possible that the timeout might expire before the query begins executing, which would cause us to hang indefinitely while waiting for the query to complete.

I'm fixing this in my own PR, but just wanted to flag this here as being a problem so that we don't do this again in the future.

Also, another problem: this probably shouldn't be using the global execution context, since its thread pool could be tied up by other uses and prevent this future from being scheduled in a timely manner.

Yes, +1 for a more reliable way to trigger cancel after submitting the long execution.

marmbrus · 2015-05-22T22:26:06Z

/cc @liancheng

SparkQA · 2015-05-26T18:13:44Z

Test build #33523 has finished for PR 6207 at commit 689e30d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

liancheng · 2015-05-27T15:31:25Z

...ftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala

+      val smallJoin = "SELECT COUNT(*) FROM test_map " +
+        List.fill(4)("join test_map").mkString(" ")
+      val rs1 = statement.executeQuery(smallJoin)
+      Await.result(sf, Duration.Inf)


Let's use a relatively long timeout (3 min?) instead of Inf here, just in case things go wrong and cause super long Jenkins build time.

liancheng · 2015-05-27T15:45:51Z

This LGTM except for some minor issues.

SparkQA · 2015-05-29T18:28:34Z

Test build #33749 has finished for PR 6207 at commit 269d5ab.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-05-30T03:18:36Z

Test build #33788 has finished for PR 6207 at commit 37bd362.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-06-05T17:09:42Z

Test build #34289 has finished for PR 6207 at commit a46b96b.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-06-05T18:29:31Z

Test build #34292 has finished for PR 6207 at commit 6ab5b28.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-06-05T19:45:54Z

Test build #34296 has finished for PR 6207 at commit 687c113.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

yhuai · 2015-06-05T19:56:13Z

jenkins test this please

SparkQA · 2015-06-05T21:39:56Z

Test build #34305 has finished for PR 6207 at commit 687c113.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yhuai · 2015-06-06T00:41:18Z

I am merging it to master.

Support runInBackground in SparkExecuteStatementOperation, and add cancellation Author: Dong Wang <dong@databricks.com> Closes #6207 from dongwang218/SPARK-6964-jdbc-cancel and squashes the following commits: 687c113 [Dong Wang] fix 100 characters 7bfa2a7 [Dong Wang] fix merge 380480f [Dong Wang] fix for liancheng's comments eb3e385 [Dong Wang] small nit 341885b [Dong Wang] small fix 3d8ebf8 [Dong Wang] add spark.sql.hive.thriftServer.async flag 04142c3 [Dong Wang] set SQLSession for async execution 184ec35 [Dong Wang] keep hive conf 819ae03 [Dong Wang] [SPARK-6964][SQL][WIP] Support Cancellation in the Thrift Server

Support runInBackground in SparkExecuteStatementOperation, and add cancellation Author: Dong Wang <dong@databricks.com> Closes apache#6207 from dongwang218/SPARK-6964-jdbc-cancel and squashes the following commits: 687c113 [Dong Wang] fix 100 characters 7bfa2a7 [Dong Wang] fix merge 380480f [Dong Wang] fix for liancheng's comments eb3e385 [Dong Wang] small nit 341885b [Dong Wang] small fix 3d8ebf8 [Dong Wang] add spark.sql.hive.thriftServer.async flag 04142c3 [Dong Wang] set SQLSession for async execution 184ec35 [Dong Wang] keep hive conf 819ae03 [Dong Wang] [SPARK-6964][SQL][WIP] Support Cancellation in the Thrift Server Conflicts: sql/hive-thriftserver/v0.13.1/src/main/scala/org/apache/spark/sql/hive/thriftserver/Shim13.scala

marmbrus · 2015-06-11T22:41:44Z

This is merged right? @dongwang218 can you close the issue?

yhuai · 2015-06-11T22:47:58Z

Yeah. I merged it. The commit is eb19d3f.

dongwang218 · 2015-06-11T22:49:10Z

yes, closing the PR, thanks @yhuai.

Support runInBackground in SparkExecuteStatementOperation, and add cancellation Author: Dong Wang <dong@databricks.com> Closes apache#6207 from dongwang218/SPARK-6964-jdbc-cancel and squashes the following commits: 687c113 [Dong Wang] fix 100 characters 7bfa2a7 [Dong Wang] fix merge 380480f [Dong Wang] fix for liancheng's comments eb3e385 [Dong Wang] small nit 341885b [Dong Wang] small fix 3d8ebf8 [Dong Wang] add spark.sql.hive.thriftServer.async flag 04142c3 [Dong Wang] set SQLSession for async execution 184ec35 [Dong Wang] keep hive conf 819ae03 [Dong Wang] [SPARK-6964][SQL][WIP] Support Cancellation in the Thrift Server

JoshRosen · 2015-12-22T00:43:16Z

...ftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala

+        statement.executeQuery(largeJoin)
+      }
+      assert(e.getMessage contains "cancelled")
+      Await.result(f, 3.minute)


Why three minutes? This seems arbitrary.

…ryServerSuite This patch fixes a flaky "test jdbc cancel" test in HiveThriftBinaryServerSuite. This test is prone to a race-condition which causes it to block indefinitely with while waiting for an extremely slow query to complete, which caused many Jenkins builds to time out. For more background, see my comments on #6207 (the PR which introduced this test). Author: Josh Rosen <joshrosen@databricks.com> Closes #10425 from JoshRosen/SPARK-11823. (cherry picked from commit 2235cd4) Signed-off-by: Josh Rosen <joshrosen@databricks.com>

…ryServerSuite This patch fixes a flaky "test jdbc cancel" test in HiveThriftBinaryServerSuite. This test is prone to a race-condition which causes it to block indefinitely with while waiting for an extremely slow query to complete, which caused many Jenkins builds to time out. For more background, see my comments on #6207 (the PR which introduced this test). Author: Josh Rosen <joshrosen@databricks.com> Closes #10425 from JoshRosen/SPARK-11823.

dongwang218 force-pushed the SPARK-6964-jdbc-cancel branch from 56251bb to 78ab6eb Compare May 20, 2015 23:17

dongwang218 changed the title ~~[SPARK-6964][SQL][WIP] Support Cancellation in the Thrift Server~~ [SPARK-6964][SQL] Support Cancellation in the Thrift Server May 21, 2015

marmbrus reviewed May 22, 2015
View reviewed changes

liancheng reviewed May 27, 2015
View reviewed changes

dongwang218 force-pushed the SPARK-6964-jdbc-cancel branch from 269d5ab to 37bd362 Compare May 30, 2015 01:21

dongwang218 force-pushed the SPARK-6964-jdbc-cancel branch from 37bd362 to a46b96b Compare June 5, 2015 17:02

Dong Wang added 4 commits June 5, 2015 11:36

[SPARK-6964][SQL][WIP] Support Cancellation in the Thrift Server

819ae03

keep hive conf

184ec35

set SQLSession for async execution

04142c3

add spark.sql.hive.thriftServer.async flag

3d8ebf8

Dong Wang added 5 commits June 5, 2015 11:36

small fix

341885b

small nit

eb3e385

fix for liancheng's comments

380480f

fix merge

7bfa2a7

fix 100 characters

687c113

dongwang218 force-pushed the SPARK-6964-jdbc-cancel branch from 6ab5b28 to 687c113 Compare June 5, 2015 18:36

dongwang218 mentioned this pull request Jun 6, 2015

[SPARK-6964][SQL] jdbc cancel 1.4 #6681

Closed

dongwang218 closed this Jun 11, 2015

JoshRosen reviewed Dec 22, 2015
View reviewed changes

JoshRosen mentioned this pull request Dec 22, 2015

[SPARK-11823][SQL] Fix flaky JDBC cancellation test in HiveThriftBinaryServerSuite #10425

Closed

[SPARK-6964][SQL] Support Cancellation in the Thrift Server #6207

[SPARK-6964][SQL] Support Cancellation in the Thrift Server #6207

Uh oh!

Conversation

dongwang218 commented May 16, 2015

Uh oh!

SparkQA commented May 16, 2015

Uh oh!

SparkQA commented May 19, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

dongwang218 commented May 21, 2015

Uh oh!

marmbrus commented May 21, 2015

Uh oh!

SparkQA commented May 22, 2015

Uh oh!

SparkQA commented May 22, 2015

Uh oh!

marmbrus May 22, 2015

Choose a reason for hiding this comment

Uh oh!

dongwang218 May 26, 2015

Choose a reason for hiding this comment

Uh oh!

JoshRosen Dec 22, 2015

Choose a reason for hiding this comment

Uh oh!

JoshRosen Dec 22, 2015

Choose a reason for hiding this comment

Uh oh!

dongwang218 Dec 22, 2015

Choose a reason for hiding this comment

Uh oh!

marmbrus commented May 22, 2015

Uh oh!

SparkQA commented May 26, 2015

Uh oh!

liancheng May 27, 2015

Choose a reason for hiding this comment

Uh oh!

liancheng commented May 27, 2015

Uh oh!

SparkQA commented May 29, 2015

Uh oh!

SparkQA commented May 30, 2015

Uh oh!

SparkQA commented Jun 5, 2015

Uh oh!

SparkQA commented Jun 5, 2015

Uh oh!

SparkQA commented Jun 5, 2015

Uh oh!

yhuai commented Jun 5, 2015

Uh oh!

SparkQA commented Jun 5, 2015

Uh oh!

yhuai commented Jun 6, 2015

Uh oh!

marmbrus commented Jun 11, 2015

Uh oh!

yhuai commented Jun 11, 2015

Uh oh!

dongwang218 commented Jun 11, 2015

Uh oh!

JoshRosen Dec 22, 2015

Choose a reason for hiding this comment

Uh oh!

Uh oh!