added basic tutorial on pyspark

README.md

@@ -24,30 +24,8 @@ PySpark Examples and Tutorials
 * cartesian: rdd1.cartesian(rdd2)
 * basic-sort: sortByKey ascending/descending
 
-How to Minimize the Verbosity of Spark
-======================================
-* Step-1: create a log4j.properties file
-````
-cp $SPARK_HOME/conf/log4j.properties.template $SPARK_HOME/conf/log4j.properties
-````
-* Step-2: Edit $SPARK_HOME/conf/log4j.properties file: replace "INFO" with "WARN"
-
-* Now your file should look like:
-````
-cat spark-1.3.0/conf/log4j.properties
-# Set everything to be logged to the console
-log4j.rootCategory=WARN, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
-
-# Settings to quiet third party logs that are too verbose
-log4j.logger.org.eclipse.jetty=WARN
-log4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERROR
-log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=WARN
-log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=WARN
-````
+[How to Minimize the Verbosity of Spark](./howto/minimize_verbosity.md) 
+=======================================================================
 
 Questions/Comments
 ==================

howto/minimize_verbosity.md

@@ -0,0 +1,24 @@
+How to Minimize the Verbosity of Spark
+======================================
+* Step-1: create a log4j.properties file
+````
+cp $SPARK_HOME/conf/log4j.properties.template $SPARK_HOME/conf/log4j.properties
+````
+* Step-2: Edit $SPARK_HOME/conf/log4j.properties file: replace "INFO" with "WARN"
+
+* Now your file should look like:
+````
+cat spark-1.3.0/conf/log4j.properties
+# Set everything to be logged to the console
+log4j.rootCategory=WARN, console
+log4j.appender.console=org.apache.log4j.ConsoleAppender
+log4j.appender.console.target=System.err
+log4j.appender.console.layout=org.apache.log4j.PatternLayout
+log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
+
+# Settings to quiet third party logs that are too verbose
+log4j.logger.org.eclipse.jetty=WARN
+log4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERROR
+log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=WARN
+log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=WARN
+````