Skip to content

Commit

Permalink
download, install spark, and run pspark
Browse files Browse the repository at this point in the history
  • Loading branch information
pyspark-in-action committed Apr 4, 2016
1 parent 6c3b8b0 commit 0c442db
Showing 1 changed file with 6 additions and 8 deletions.
14 changes: 6 additions & 8 deletions howto/download_install_run_spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,24 +132,22 @@ SparkContext available as sc, HiveContext available as sqlContext.
['john', 'paul', 'george', 'ringo']
>>> testScriptPath = "/Users/mparsian/spark-1.6.1-bin-hadoop2.6/test.py"
>>> test2ScriptPath = "/Users/mparsian/spark-1.6.1-bin-hadoop2.6/test2.py"
>>> import testScriptPath
>>> import test2ScriptPath
>>> test = "/Users/mparsian/spark-1.6.1-bin-hadoop2.6/test.py"
>>> test2 = "/Users/mparsian/spark-1.6.1-bin-hadoop2.6/test2.py"
>>> import test
>>> import test2
>>> pipeRDD = rdd.pipe(testScriptPath)
>>> pipeRDD = rdd.pipe(test)
>>> pipeRDD.collect()
[u'hello john', u'', u'hello paul', u'', u'hello george', u'', u'hello ringo', u'']
>>> rdd.collect()
['john', 'paul', 'george', 'ringo']
>>> rdd2 = rdd.map(lambda x : script2.func(x))
>>> rdd2.collect()
>>> rdd2 = rdd.map(lambda x : test2ScriptPath.fun2(x))
>>> rdd2 = rdd.map(lambda x : test2.fun2(x))
>>> rdd2.collect()
['john zaza', 'paul zaza', 'george zaza', 'ringo zaza']
>>>
Expand Down

0 comments on commit 0c442db

Please sign in to comment.