Skip to content

Commit ea66228

Browse files
zjffdudavies
authored andcommitted
[SPARK-17261] [PYSPARK] Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"
## What changes were proposed in this pull request? Set SparkSession._instantiatedContext as None so that we can recreate SparkSession again. ## How was this patch tested? Tested manually using the following command in pyspark shell ``` spark.stop() spark = SparkSession.builder.enableHiveSupport().getOrCreate() spark.sql("show databases").show() ``` Author: Jeff Zhang <zjffdu@apache.org> Closes #14857 from zjffdu/SPARK-17261.
1 parent 6bcbf9b commit ea66228

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

python/pyspark/sql/session.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -595,6 +595,7 @@ def stop(self):
595595
"""Stop the underlying :class:`SparkContext`.
596596
"""
597597
self._sc.stop()
598+
SparkSession._instantiatedContext = None
598599

599600
@since(2.0)
600601
def __enter__(self):

0 commit comments

Comments
 (0)