[SPARK-2851] [mllib] DecisionTree Python consistency update #1798

jkbradley · 2014-08-06T02:39:21Z

Added 6 static train methods to match Python API, but without default arguments (but with Python default args noted in docs).

Added factory classes for Algo and Impurity, but made private[mllib].

CC: @mengxr @dorx Please let me know if there are other changes which would help with API consistency---thanks!

…thout default parameters

* Added 6 static train methods to match Python API, but without default arguments (but with Python default args noted in docs). Added factory classes for Algo and Impurity, but made private[mllib].

SparkQA · 2014-08-06T02:44:30Z

QA tests have started for PR 1798. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17980/consoleFull

mengxr · 2014-08-06T02:48:55Z

mllib/src/main/scala/org/apache/spark/mllib/tree/configuration/Algo.scala

@@ -27,4 +27,10 @@ import org.apache.spark.annotation.Experimental
 object Algo extends Enumeration {
  type Algo = Value
  val Classification, Regression = Value
+
+  private[mllib] def stringToAlgo(name: String): Algo = name match {


minor: stringToAlgo -> fromString? In the code, it might be more natural to read

Algo.fromString("classification")

Algo.stringToAlgo("classification")

SparkQA · 2014-08-06T03:45:03Z

QA results for PR 1798:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17980/consoleFull

…ency

* Removed train() function in Python API (tree.py) ** Removed corresponding function in Scala/Java API (the ones taking basic types) DecisionTree internal updates: * Renamed Algo and Impurity factory methods to fromString() DecisionTree doc updates: * Added notes recommending use of trainClassifier, trainRegressor * Say supported values for impurity * Shortened doc for Java-friendly train* functions.

SparkQA · 2014-08-06T18:49:39Z

QA tests have started for PR 1798. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18037/consoleFull

SparkQA · 2014-08-06T19:42:02Z

QA results for PR 1798:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18037/consoleFull

mengxr · 2014-08-06T19:47:44Z

mllib/src/main/scala/org/apache/spark/mllib/tree/DecisionTree.scala

+   * Java-friendly API for [[org.apache.spark.mllib.tree.DecisionTree$#trainClassifier]]
+   */
+  def trainClassifier(
+      input: RDD[LabeledPoint],


RDD -> JavaRDD. Sorry I missed this in the first pass.

mengxr · 2014-08-06T19:48:06Z

LGTM except minor inline comments.

… instead of RDD.

jkbradley · 2014-08-06T20:29:20Z

Hopefully all done now, after Jenkins.

SparkQA · 2014-08-06T20:34:22Z

QA tests have started for PR 1798. This patch DID NOT merge cleanly!
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18044/consoleFull

mengxr · 2014-08-06T21:07:11Z

@jkbradley Could you try to merge the latest master? It seems that there are conflicts.

…ency Conflicts: python/pyspark/mllib/tree.py (not a real conflict, merged)

SparkQA · 2014-08-06T21:39:31Z

QA tests have started for PR 1798. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18054/consoleFull

SparkQA · 2014-08-06T21:54:43Z

QA results for PR 1798:
- This patch PASSES unit tests.

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18044/consoleFull

SparkQA · 2014-08-06T22:31:49Z

QA results for PR 1798:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18054/consoleFull

Added 6 static train methods to match Python API, but without default arguments (but with Python default args noted in docs). Added factory classes for Algo and Impurity, but made private[mllib]. CC: mengxr dorx Please let me know if there are other changes which would help with API consistency---thanks! Author: Joseph K. Bradley <joseph.kurata.bradley@gmail.com> Closes #1798 from jkbradley/dt-python-consistency and squashes the following commits: 6f7edf8 [Joseph K. Bradley] Merge remote-tracking branch 'upstream/master' into dt-python-consistency a0d7dbe [Joseph K. Bradley] DecisionTree: In Java-friendly train* methods, changed to use JavaRDD instead of RDD. ee1d236 [Joseph K. Bradley] DecisionTree API updates: * Removed train() function in Python API (tree.py) ** Removed corresponding function in Scala/Java API (the ones taking basic types) 00f820e [Joseph K. Bradley] Merge remote-tracking branch 'upstream/master' into dt-python-consistency fe6dbfa [Joseph K. Bradley] removed unnecessary imports e358661 [Joseph K. Bradley] DecisionTree API change: * Added 6 static train methods to match Python API, but without default arguments (but with Python default args noted in docs). c699850 [Joseph K. Bradley] a few doc comments eaf84c0 [Joseph K. Bradley] Added DecisionTree static train() methods API to match Python, but without default parameters (cherry picked from commit 47ccd5e) Signed-off-by: Xiangrui Meng <meng@databricks.com>

mengxr · 2014-08-07T06:00:43Z

LGTM. Merged into both master and branch-1.1.

Added 6 static train methods to match Python API, but without default arguments (but with Python default args noted in docs). Added factory classes for Algo and Impurity, but made private[mllib]. CC: mengxr dorx Please let me know if there are other changes which would help with API consistency---thanks! Author: Joseph K. Bradley <joseph.kurata.bradley@gmail.com> Closes apache#1798 from jkbradley/dt-python-consistency and squashes the following commits: 6f7edf8 [Joseph K. Bradley] Merge remote-tracking branch 'upstream/master' into dt-python-consistency a0d7dbe [Joseph K. Bradley] DecisionTree: In Java-friendly train* methods, changed to use JavaRDD instead of RDD. ee1d236 [Joseph K. Bradley] DecisionTree API updates: * Removed train() function in Python API (tree.py) ** Removed corresponding function in Scala/Java API (the ones taking basic types) 00f820e [Joseph K. Bradley] Merge remote-tracking branch 'upstream/master' into dt-python-consistency fe6dbfa [Joseph K. Bradley] removed unnecessary imports e358661 [Joseph K. Bradley] DecisionTree API change: * Added 6 static train methods to match Python API, but without default arguments (but with Python default args noted in docs). c699850 [Joseph K. Bradley] a few doc comments eaf84c0 [Joseph K. Bradley] Added DecisionTree static train() methods API to match Python, but without default parameters

jkbradley added 4 commits August 5, 2014 16:36

Added DecisionTree static train() methods API to match Python, but wi…

eaf84c0

…thout default parameters

a few doc comments

c699850

DecisionTree API change:

e358661

* Added 6 static train methods to match Python API, but without default arguments (but with Python default args noted in docs). Added factory classes for Algo and Impurity, but made private[mllib].

removed unnecessary imports

fe6dbfa

jkbradley changed the title ~~DecisionTree Python consistency update~~ [mllib] DecisionTree Python consistency update Aug 6, 2014

jkbradley changed the title ~~[mllib] DecisionTree Python consistency update~~ [SPARK-2828] [mllib] DecisionTree Python consistency update Aug 6, 2014

jkbradley changed the title ~~[SPARK-2828] [mllib] DecisionTree Python consistency update~~ [SPARK-2851] [mllib] DecisionTree Python consistency update Aug 6, 2014

mengxr reviewed Aug 6, 2014
View reviewed changes

jkbradley added 2 commits August 5, 2014 23:19

Merge remote-tracking branch 'upstream/master' into dt-python-consist…

00f820e

…ency

mengxr reviewed Aug 6, 2014
View reviewed changes

DecisionTree: In Java-friendly train* methods, changed to use JavaRDD…

a0d7dbe

… instead of RDD.

Merge remote-tracking branch 'upstream/master' into dt-python-consist…

6f7edf8

…ency Conflicts: python/pyspark/mllib/tree.py (not a real conflict, merged)

asfgit closed this in 47ccd5e Aug 7, 2014

jkbradley deleted the dt-python-consistency branch December 4, 2014 20:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SPARK-2851] [mllib] DecisionTree Python consistency update #1798

[SPARK-2851] [mllib] DecisionTree Python consistency update #1798

Uh oh!

jkbradley commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr Aug 6, 2014

Uh oh!

mengxr commented Aug 6, 2014

Uh oh!

jkbradley commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr commented Aug 7, 2014

Uh oh!

Uh oh!

[SPARK-2851] [mllib] DecisionTree Python consistency update #1798

[SPARK-2851] [mllib] DecisionTree Python consistency update #1798

Uh oh!

Conversation

jkbradley commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr Aug 6, 2014

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr Aug 6, 2014

Choose a reason for hiding this comment

Uh oh!

mengxr commented Aug 6, 2014

Uh oh!

jkbradley commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

SparkQA commented Aug 6, 2014

Uh oh!

mengxr commented Aug 7, 2014

Uh oh!

Uh oh!