Skip to content

Test flake: org.apache.spark.sql.execution.datasources.json.JsonSuite #123

Closed
@ash211

Description

@ash211

https://circleci.com/gh/palantir/spark/460#artifacts/containers/2

18987 [info] *** 1 SUITE ABORTED ***
18988 [error] Error: Total 3021, Failed 0, Errors 1, Passed 3020, Ignored 55, Canceled 1
18989 [error] Error during tests:
18990 [error]     org.apache.spark.sql.execution.datasources.json.JsonSuite
16438 [info] - SPARK-18352: Handle multi-line corrupt documents (FAILFAST) (1 second, 437 milliseconds)
16439 00:34:46.858 WARN org.apache.spark.scheduler.TaskSetManager: Lost task 1.0 in stage 318.0 (TID 624, localhost, executor driver): TaskKilled (killed intentionally)
16440 [info] Exception encountered when attempting to run a suite with class name: org.apache.spark.sql.execution.datasources.json.JsonSuite *** ABORTED *** (11 seconds, 356 milliseconds)
16441 [info]   java.lang.RuntimeException: There are 1 possibly leaked file streams.
16442 [info]   at org.apache.spark.DebugFilesystem$.assertNoOpenStreams(DebugFilesystem.scala:47)
16443 [info]   at org.apache.spark.sql.test.SharedSQLContext$class.afterEach(SharedSQLContext.scala:87)
16444 [info]   at org.apache.spark.sql.execution.datasources.json.JsonSuite.afterEach(JsonSuite.scala:45)
16445 [info]   at org.scalatest.BeforeAndAfterEach$class.afterEach(BeforeAndAfterEach.scala:205)
16446 [info]   at org.apache.spark.sql.execution.datasources.json.JsonSuite.afterEach(JsonSuite.scala:45)
16447 [info]   at org.scalatest.BeforeAndAfterEach$class.afterEach(BeforeAndAfterEach.scala:220)
16448 [info]   at org.apache.spark.sql.execution.datasources.json.JsonSuite.afterEach(JsonSuite.scala:45)
16449 [info]   at org.scalatest.BeforeAndAfterEach$class.runTest(BeforeAndAfterEach.scala:264)
16450 [info]   at org.apache.spark.sql.execution.datasources.json.JsonSuite.runTest(JsonSuite.scala:45)
16451 [info]   at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
16452 [info]   at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
16453 [info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
16454 [info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
16455 [info]   at scala.collection.immutable.List.foreach(List.scala:381)
16456 [info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
16457 [info]   at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
16458 [info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
16459 [info]   at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:208)
16460 [info]   at org.scalatest.FunSuite.runTests(FunSuite.scala:1555)
16461 [info]   at org.scalatest.Suite$class.run(Suite.scala:1424)
16462 [info]   at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1555)
16463 [info]   at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
16464 [info]   at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
16465 [info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
16466 [info]   at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:212)
16467 [info]   at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:31)
16468 [info]   at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:257)
16469 [info]   at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:256)
16470 [info]   at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:31)
16471 [info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:357)
16472 [info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:502)
16473 [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:296)
16474 [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:286)
16475 [info]   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
16476 [info]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
16477 [info]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
16478 [info]   at java.lang.Thread.run(Thread.java:745)

Looks like a file stream got leaked somewhere?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions