Skip to content

Commit 18c8723

Browse files
committed
modified streaming test case to add coment
1 parent 13fb44c commit 18c8723

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

python/pyspark/streaming_tests.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@
1818
"""
1919
Unit tests for PySpark; additional tests are implemented as doctests in
2020
individual modules.
21+
Other option is separate this test case with other tests.
22+
This makes sense becuase streaming tests takes long time due to waiting time
23+
for stoping callback server.
2124
2225
This file will merged to tests.py. But for now, this file is separated due
2326
to focusing to streaming test case
@@ -45,7 +48,7 @@ def tearDown(self):
4548
self.ssc._sc.stop()
4649
# Why does it long time to terminaete StremaingContext and SparkContext?
4750
# Should we change the sleep time if this depends on machine spec?
48-
time.sleep(8)
51+
time.sleep(10)
4952

5053
@classmethod
5154
def tearDownClass(cls):
@@ -302,7 +305,7 @@ def _run_stream(self, test_input, test_func, expected_output, numSlices=None):
302305
"""Start stream and return the output"""
303306
# Generate input stream with user-defined input
304307
numSlices = numSlices or self.numInputPartitions
305-
test_input_stream = self.ssc._testInputStream2(test_input, numSlices)
308+
test_input_stream = self.ssc._testInputStream(test_input, numSlices)
306309
# Apply test function to stream
307310
test_stream = test_func(test_input_stream)
308311
# Add job to get output from stream

0 commit comments

Comments
 (0)