[SPARK-13655] [STREAMING] [TESTS] Fix WithAggregationKinesisBackedBlockRDDSuite

[srowen]

What changes were proposed in this pull request?

Disable Kinesis tests that are always timing out the build. Anything triggering kinesis tests is currently hanging and timing out, like #11481

This does not resolve the problem, just disables the test.

CC @brkyvz @tdas who may know more, having mostly authored / changed this code.

How was this patch tested?

Jenkins tests. As this only removes a test, it shouldn't be able to break anything. We'll see if this was all that's needed.

[SparkQA]

Test build #52551 has finished for PR 11558 at commit d1261fa.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Great! Thank you for trying to resolve this!

[SparkQA]

Test build #52581 has finished for PR 11558 at commit c3ce229.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[JoshRosen]

@tdas @brkyvz, don't these tests run against a real kinesis or something like that? Maybe there's a misconfiguration of credentials in Jenkins? If you remember how credentials were originally set up then forward those details to me and I can try to port that setup onto new build / jenkins scripts in case that's the issue.

[zsxwing]

@JoshRosen This one was not tested with your block manager changes :(.

I can reproduce the timeout failure locally. Here is the stack trace:

"pool-1-thread-1-ScalaTest-running-WithAggregationKinesisBackedBlockRDDSuite" #11 prio=5 os_prio=31 tid=0x00007fe519242800 nid=0x5703 in Object.wait() [0x0000000130000000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:236)
    - locked <0x0000000700f35cd8> (a org.apache.spark.storage.BlockInfoManager)
    at org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1126)
    at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests$$anonfun$org$apache$spark$streaming$kinesis$KinesisBackedBlockRDDTests$$testRDD$7.apply(KinesisBackedBlockRDDSuite.scala:172)
    at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests$$anonfun$org$apache$spark$streaming$kinesis$KinesisBackedBlockRDDTests$$testRDD$7.apply(KinesisBackedBlockRDDSuite.scala:172)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
    at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests.org$apache$spark$streaming$kinesis$KinesisBackedBlockRDDTests$$testRDD(KinesisBackedBlockRDDSuite.scala:172)
    at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests$$anonfun$3.apply$mcV$sp(KinesisBackedBlockRDDSuite.scala:105)
    at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests$$anonfun$3.apply(KinesisBackedBlockRDDSuite.scala:105)
    at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests$$anonfun$3.apply(KinesisBackedBlockRDDSuite.scala:105)

[zsxwing]

@JoshRosen you can test it using ENABLE_KINESIS_TESTS=1 sbt/sbt -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Pkinesis-asl "project streaming-kinesis-asl" "test-only *WithAggregationKinesisBackedBlockRDDSuite" with your credentials.

[JoshRosen]

@zsxwing, thanks for the pointer. I think the problem here is that this suite is sharing a blockmanager and block ids across test cases and some of the test cases directly call BlockManager methods outside of a task to get blocks and then do not release those locks.

What do you think about refactoring this suite so that the state is completely reset between tests?

[JoshRosen]

Here's an alternate fix: #11564

[JoshRosen]

@srowen, we merged my fix in #11564 so I think you can close this noe.

[srowen]

Nice! actual fix.