[SPARK-7387][ml][doc] CrossValidator example code in Python

[harsha2010]

No description provided.

[harsha2010]

@jkbradley @mengxr, please review. I wanted to add an example of how to introspect the best model's parameters in python (in scala, i was able to get the pipeline model's stages and extract the hashingTF.numFeatures and lrModel.regParam). But the Python implementation doesn't seem to expose lrModel.getParam (i might be mistaken here)

[SparkQA]

Test build #33344 has finished for PR 6358 at commit 7246d35.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

@harsha2010 You're correct that the Python API is incomplete. Most of the spark.ml Models are simple wrappers around Java objects and don't expose much functionality other than transform() right now. Next release...

[jkbradley]

This does not give the hoped-for predictions on the test data (predicting everything with "Spark" as 1.0). In the Scala and Java examples, I added more training examples to make it more likely that the chosen model would look for "Spark." Could you please add those other training examples here too?

[harsha2010]

@jkbradley , sure..is this checked into master yet? on master i notice the most recent change was to change the regularization and one of the test examples

[jkbradley]

Yep, it's from Spark 1.3: [https://github.com/apache/spark/blob/1bb5d716c0351cd0b4c11b397fd778f30db39bd9/examples/src/main/scala/org/apache/spark/examples/ml/CrossValidatorExample.scala]

[jkbradley]

@harsha2010 Also, would you mind updating (either in this PR or another) the forest & GBT examples you just wrote? After a little investigation, I realized the explanation I gave for why Spark SQL can infer the schema from LabeledPoint is wrong. It's because LabeledPoint is an Object, and Spark SQL identifies the 2 fields "label" and "features" from the Object.

[SparkQA]

Test build #33971 has finished for PR 6358 at commit 54a500c.

• This patch fails Python style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #33975 has finished for PR 6358 at commit aeb6bb6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

Except for the issue with needing more training examples, this looks good to me.

[harsha2010]

@jkbradley thanks, i have updated the cross validator with enough examples now

[jkbradley]

@harsha2010 Thanks! (Does it give the hoped-for predictions on the test data now when you run it?)

[harsha2010]

@jkbradley yes it does much better now

Row(id=4, text=u'spark i j k', probability=DenseVector([0.248, 0.752]), prediction=1.0)
Row(id=5, text=u'l m n', probability=DenseVector([0.9647, 0.0353]), prediction=0.0)
Row(id=6, text=u'mapreduce spark', probability=DenseVector([0.4248, 0.5752]), prediction=1.0)
Row(id=7, text=u'apache hadoop', probability=DenseVector([0.69, 0.31]), prediction=0.0)

[jkbradley]

Haha good. LGTM pending tests

[SparkQA]

Test build #34008 has finished for PR 6358 at commit 63efda2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

Merging with master and branch-1.4