[SPARK-3458] enable python "with" statements for SparkContext #2335
Conversation
allow for best practice code, try: sc = SparkContext() app(sc) finally: sc.stop() to be written using a "with" statement, with SparkContext() as sc: app(sc)
|
@davies i'd appreciate your input on this |
|
QA tests have started for PR 2335 at commit
|
| sc = SparkContext() | ||
| self.assertNotEqual(SparkContext._active_spark_context, None) | ||
| sc.stop() | ||
| self.assertEqual(SparkContext._active_spark_context, None) |
There was a problem hiding this comment.
self.assertIsNone(xxx) is better than self.assertEqual(xxx, None)
There was a problem hiding this comment.
accordning to https://docs.python.org/2/library/unittest.html#assert-methods assertIsNone and assertIsNotNone are new to python 2.7 and my (potentially incorrect) understanding is we want to target 2.6.
thoughts?
There was a problem hiding this comment.
Sorry, I missed that, we definitely need to target Python2.6.
There was a problem hiding this comment.
side note: i can't even find a box w/ py2.6 (it's so old!)
There was a problem hiding this comment.
Yeah, I know :( . Unfortunately, Python 2.6 is still the default Python version on a lot of Linux distributions, so we've been trying to maintain backwards-compatibility in order to ensure a good out-of-the-box experience for those users.
|
@mattf It's cool to have this, just one minor comment about tests. |
|
QA tests have finished for PR 2335 at commit
|
failures are unrelated to this patch |
|
Jenkins, retest this please. |
|
I'm OK to merge this. |
|
QA tests have started for PR 2335 at commit
|
|
QA tests have finished for PR 2335 at commit
|
still not this patch's fault |
|
Jenkins, retest this please. |
|
QA tests have started for PR 2335 at commit
|
|
QA tests have finished for PR 2335 at commit
|
|
Merging in master. Thanks! |
allow for best practice code,
to be written using a "with" statement,