[SPARK-3516] [mllib] DecisionTree: Add minInstancesPerNode, minInfoGain params to example and Python API #2349

jkbradley · 2014-09-10T23:53:39Z

Added minInstancesPerNode, minInfoGain params to:

DecisionTreeRunner.scala example
Python API (tree.py)

Also:

Fixed typo in tree suite test "do not choose split that does not satisfy min instance per node requirements"
small style fixes

CC: @mengxr

…rune Conflicts: mllib/src/main/scala/org/apache/spark/mllib/tree/configuration/Strategy.scala

…unner.scala and to Python API in tree.py

…eprune

…tisfy min instance per node requirements" * small style fixes

SparkQA · 2014-09-11T00:24:03Z

QA tests have started for PR 2349 at commit 95c479d.

This patch merges cleanly.

SparkQA · 2014-09-11T00:53:23Z

QA tests have started for PR 2349 at commit 95c479d.

This patch merges cleanly.

SparkQA · 2014-09-11T01:25:29Z

QA tests have finished for PR 2349 at commit 95c479d.

This patch passes unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2014-09-11T02:03:45Z

QA tests have finished for PR 2349 at commit 95c479d.

This patch passes unit tests.
This patch merges cleanly.
This patch adds no public classes.

davies · 2014-09-11T04:47:09Z

mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala

Why "don't" is typo?

Not really a typo. But I figured that, if people are munging logs from tests, quote characters might be troublesome to deal with.

It sounds reasonable, thanks.

davies · 2014-09-11T04:47:46Z

This patch looks good to me, just one minor question.

jkbradley · 2014-09-11T05:40:20Z

@davies Thanks for taking a look!

mengxr · 2014-09-12T16:43:11Z

@jkbradley This contains API changes to python. Could you create a JIRA for it? Thanks!

Conflicts: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala

SparkQA · 2014-09-13T00:44:18Z

QA tests have started for PR 2349 at commit a95e7c8.

This patch merges cleanly.

SparkQA · 2014-09-13T01:04:17Z

QA tests have started for PR 2349 at commit 61b2e72.

This patch merges cleanly.

SparkQA · 2014-09-13T01:39:41Z

QA tests have finished for PR 2349 at commit a95e7c8.

This patch fails unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- throw new IllegalStateException("The main method in the given main class must be static")

jkbradley · 2014-09-13T01:42:17Z

Unrelated failure (in streaming)

SparkQA · 2014-09-13T02:03:54Z

QA tests have finished for PR 2349 at commit 61b2e72.

This patch passes unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- throw new IllegalStateException("The main method in the given main class must be static")

jkbradley · 2014-09-13T20:36:11Z

@mengxr This patch should be ready to check now. Thanks!

mengxr · 2014-09-16T00:43:52Z

LGTM. Merged into master. Thanks!

qiping.lqp and others added 21 commits September 9, 2014 11:17

add min info gain and min instances per node parameters in decision tree

ac42378

separate calculation of predict of node from calculation of info gain

ff34845

fix bug

987cbf4

fix style

f195e83

fix style

845c6fa

add comments

e72c7e4

fix bug

46b891f

add api docs

cadd569

Merge branch 'master' of https://github.com/apache/spark into dt-prep…

bb465ca

…rune Conflicts: mllib/src/main/scala/org/apache/spark/mllib/tree/configuration/Strategy.scala

minor fix: remove empty lines

6728fad

fix style

10b8012

fix bug

efcc736

fix docs and change minInstancesPerNode to 1

d593ec7

remove noSplit and set Predict private to tree

0278a11

Added minInstancesPerNode and minInfoGain parameters to DecisionTreeR…

c6e2dfc

…unner.scala and to Python API in tree.py

change edge minInstancesPerNode to 2 and add one more test

39f9b60

fix typo

c7ebaf1

fix typo

f1d11d1

Merge remote-tracking branch 'chouqin/dt-preprune' into chouqin-dt-pr…

19b01af

…eprune

Merge remote-tracking branch 'upstream/master' into chouqin-dt-preprune

e2628b6

* Fixed typo in tree suite test "do not choose split that does not sa…

95c479d

…tisfy min instance per node requirements" * small style fixes

davies reviewed Sep 11, 2014
View reviewed changes

Merge remote-tracking branch 'upstream/master' into chouqin-dt-preprune

a95e7c8

Conflicts: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala

Added max of 10GB for maxMemoryInMB in Strategy.

61b2e72

jkbradley changed the title ~~[mllib] DecisionTree: Add minInstancesPerNode, minInfoGain params to example and Python API~~ [SPARK-3516] [mllib] DecisionTree: Add minInstancesPerNode, minInfoGain params to example and Python API Sep 13, 2014

asfgit closed this in fdb302f Sep 16, 2014

jkbradley deleted the chouqin-dt-preprune branch October 8, 2014 21:20

[SPARK-3516] [mllib] DecisionTree: Add minInstancesPerNode, minInfoGain params to example and Python API #2349

[SPARK-3516] [mllib] DecisionTree: Add minInstancesPerNode, minInfoGain params to example and Python API #2349

Uh oh!

Conversation

jkbradley commented Sep 10, 2014

Uh oh!

SparkQA commented Sep 11, 2014

Uh oh!

SparkQA commented Sep 11, 2014

Uh oh!

SparkQA commented Sep 11, 2014

Uh oh!

SparkQA commented Sep 11, 2014

Uh oh!

davies Sep 11, 2014

Choose a reason for hiding this comment

Uh oh!

jkbradley Sep 11, 2014

Choose a reason for hiding this comment

Uh oh!

davies Sep 11, 2014

Choose a reason for hiding this comment

Uh oh!

davies commented Sep 11, 2014

Uh oh!

jkbradley commented Sep 11, 2014

Uh oh!

mengxr commented Sep 12, 2014

Uh oh!

SparkQA commented Sep 13, 2014

Uh oh!

SparkQA commented Sep 13, 2014

Uh oh!

SparkQA commented Sep 13, 2014

Uh oh!

jkbradley commented Sep 13, 2014

Uh oh!

SparkQA commented Sep 13, 2014

Uh oh!

jkbradley commented Sep 13, 2014

Uh oh!

mengxr commented Sep 16, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants