Skip to content

Added samples for tree regression trainers. #2999

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 19, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.ML.Data;

namespace Microsoft.ML.Samples.Dynamic.Trainers.Regression
{
public static class FastForest
{
// This example requires installation of additional NuGet package
// <a href="https://www.nuget.org/packages/Microsoft.ML.FastTree/">Microsoft.ML.FastTree</a>.
Copy link
Member

@wschin wschin Mar 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be xml doc. #Resolved

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xml looks busy, and it won't render as xml inside the example. This code goes in the example block.


In reply to: 266665386 [](ancestors = 266665386)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// xml are not used in .net examples, so we're sticking to // comments inside samples' code.


In reply to: 266722954 [](ancestors = 266722954,266665386)

public static void Example()
{
// Create a new context for ML.NET operations. It can be used for exception tracking and logging,
// as a catalog of available operations and as the source of randomness.
// Setting the seed to a fixed number in this example to make outputs deterministic.
var mlContext = new MLContext(seed: 0);

// Create a list of training examples.
var examples = GenerateRandomDataPoints(1000);

// Convert the examples list to an IDataView object, which is consumable by ML.NET API.
var trainingData = mlContext.Data.LoadFromEnumerable(examples);

// Define the trainer.
var pipeline = mlContext.Regression.Trainers.FastForest();
Copy link
Member

@wschin wschin Mar 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe user wants to learn some little configuration. Could you somehow make it non-default? #Resolved

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had thought about this, and decided to keep it as defaults for the following reasons:

  1. ML.NET has done some work with tuning the defaults so that users can have a good starting point. If we add parameters here, users would think that they have to worry about coming up with those parameters for their own scenario. And that's a negative. It's much easier for users to think that they don't need any parameter, until they actually know what they're doing.

  2. Any .NET developer can easily find out what parameters are available. The real value is not to show them the parameters, but to explain why they might need to change say number of trees or leaves. We're already doing that in the parameter description, and it would be too much to go over those details in the sample body.


In reply to: 266665195 [](ancestors = 266665195)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is another example in FastForestWithOptions.cs that shows the configuration.


In reply to: 267017735 [](ancestors = 267017735,266665195)


// Train the model.
var model = pipeline.Fit(trainingData);

// Create testing examples. Use different random seed to make it different from training data.
var testData = mlContext.Data.LoadFromEnumerable(GenerateRandomDataPoints(500, seed:123));
Copy link
Member

@sfilipi sfilipi Mar 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seed:123 [](start = 91, length = 8)

why are you setting the seed here, but not above? #Resolved

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've explained it in the comment line above. We want training and test set to be different. The first one uses default seed of 0. Here I use a different seed.


In reply to: 266723117 [](ancestors = 266723117)

Copy link
Contributor

@zeahmed zeahmed Mar 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can we be explicit to show the seed:0 in the training case as well. Also, can seed:123 be changed to seed:1? or maybe I should better ask if there is any significance of using seed:123?


In reply to: 267018416 [](ancestors = 267018416,266723117)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we're splitting hairs here. 123 is just a random number like saying ABC.


In reply to: 267029054 [](ancestors = 267029054,267018416,266723117)


// Run the model on test data set.
var transformedTestData = model.Transform(testData);

// Convert IDataView object to a list.
var predictions = mlContext.Data.CreateEnumerable<Prediction>(transformedTestData, reuseRowObject: false).ToList();

// Look at 5 predictions
foreach (var p in predictions.Take(5))
Console.WriteLine($"Label: {p.Label:F3}, Prediction: {p.Score:F3}");

// Expected output:
// Label: 0.985, Prediction: 0.864
// Label: 0.155, Prediction: 0.164
// Label: 0.515, Prediction: 0.470
// Label: 0.566, Prediction: 0.501
// Label: 0.096, Prediction: 0.138

// Evaluate the overall metrics
var metrics = mlContext.Regression.Evaluate(transformedTestData);
SamplesUtils.ConsoleUtils.PrintMetrics(metrics);

// Expected output:
// Mean Absolute Error: 0.06
// Mean Squared Error: 0.01
// Root Mean Squared Error: 0.07
// RSquared: 0.93
}

private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count, int seed=0)
{
var random = new Random(seed);
float randomFloat() => (float)random.NextDouble();
for (int i = 0; i < count; i++)
{
var label = randomFloat();
yield return new DataPoint
{
Label = label,
// Create random features that are correlated with label.
Features = Enumerable.Repeat(label, 50).Select(x => x + randomFloat()).ToArray()
};
}
}

// Example with label and 50 feature values. A data set is a collection of such examples.
private class DataPoint
{
public float Label { get; set; }
[VectorType(50)]
public float[] Features { get; set; }
}

// Class used to capture predictions.
private class Prediction
{
// Original label.
public float Label { get; set; }
// Predicted score from the trainer.
public float Score { get; set; }
}
Copy link
Contributor

@zeahmed zeahmed Mar 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to be common piece of code in all the example (I assume). Did we think of templating it in some way so that in future if need to change it we can change it at one place? #Resolved

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're adding boilerplate code in order to make the samples self-contained without dependency on external files that users have to look up. please see #2726.


In reply to: 267029777 [](ancestors = 267029777)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean this: #3017
Obviously not going to be in this PR.


In reply to: 267038999 [](ancestors = 267038999,267029777)

}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.ML.Data;
using Microsoft.ML.Trainers.FastTree;

namespace Microsoft.ML.Samples.Dynamic.Trainers.Regression
{
public static class FastForestWithOptions
{
// This example requires installation of additional NuGet package
// <a href="https://www.nuget.org/packages/Microsoft.ML.FastTree/">Microsoft.ML.FastTree</a>.
public static void Example()
{
// Create a new context for ML.NET operations. It can be used for exception tracking and logging,
// as a catalog of available operations and as the source of randomness.
// Setting the seed to a fixed number in this example to make outputs deterministic.
var mlContext = new MLContext(seed: 0);

// Create a list of training examples.
var examples = GenerateRandomDataPoints(1000);

// Convert the examples list to an IDataView object, which is consumable by ML.NET API.
var trainingData = mlContext.Data.LoadFromEnumerable(examples);

// Define trainer options.
var options = new FastForestRegressionTrainer.Options
{
// Only use 80% of features to reduce over-fitting.
FeatureFraction = 0.8,
// Create a simpler model by penalizing usage of new features.
FeatureFirstUsePenalty = 0.1,
// Reduce the number of trees to 50.
NumberOfTrees = 50
};

// Define the trainer.
var pipeline = mlContext.Regression.Trainers.FastForest(options);

// Train the model.
var model = pipeline.Fit(trainingData);

// Create testing examples. Use different random seed to make it different from training data.
var testData = mlContext.Data.LoadFromEnumerable(GenerateRandomDataPoints(500, seed:123));

// Run the model on test data set.
var transformedTestData = model.Transform(testData);

// Convert IDataView object to a list.
var predictions = mlContext.Data.CreateEnumerable<Prediction>(transformedTestData, reuseRowObject: false).ToList();

// Look at 5 predictions
foreach (var p in predictions.Take(5))
Console.WriteLine($"Label: {p.Label:F3}, Prediction: {p.Score:F3}");

// Expected output:
// Label: 0.985, Prediction: 0.866
// Label: 0.155, Prediction: 0.171
// Label: 0.515, Prediction: 0.470
// Label: 0.566, Prediction: 0.476
// Label: 0.096, Prediction: 0.140

// Evaluate the overall metrics
var metrics = mlContext.Regression.Evaluate(transformedTestData);
SamplesUtils.ConsoleUtils.PrintMetrics(metrics);

// Expected output:
// Mean Absolute Error: 0.06
// Mean Squared Error: 0.01
// Root Mean Squared Error: 0.08
// RSquared: 0.93
}

private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count, int seed=0)
{
var random = new Random(seed);
float randomFloat() => (float)random.NextDouble();
for (int i = 0; i < count; i++)
{
var label = randomFloat();
yield return new DataPoint
{
Label = label,
// Create random features that are correlated with label.
Features = Enumerable.Repeat(label, 50).Select(x => x + randomFloat()).ToArray()
};
}
}

// Example with label and 50 feature values. A data set is a collection of such examples.
private class DataPoint
{
public float Label { get; set; }
[VectorType(50)]
public float[] Features { get; set; }
}

// Class used to capture predictions.
private class Prediction
{
// Original label.
public float Label { get; set; }
// Predicted score from the trainer.
public float Score { get; set; }
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.ML.Data;

namespace Microsoft.ML.Samples.Dynamic.Trainers.Regression
{
public static class FastTreeTweedie
{
// This example requires installation of additional NuGet package
// <a href="https://www.nuget.org/packages/Microsoft.ML.FastTree/">Microsoft.ML.FastTree</a>.
public static void Example()
{
// Create a new context for ML.NET operations. It can be used for exception tracking and logging,
// as a catalog of available operations and as the source of randomness.
// Setting the seed to a fixed number in this example to make outputs deterministic.
var mlContext = new MLContext(seed: 0);

// Create a list of training examples.
var examples = GenerateRandomDataPoints(1000);

// Convert the examples list to an IDataView object, which is consumable by ML.NET API.
var trainingData = mlContext.Data.LoadFromEnumerable(examples);

// Define the trainer.
var pipeline = mlContext.Regression.Trainers.FastTreeTweedie();

// Train the model.
var model = pipeline.Fit(trainingData);

// Create testing examples. Use different random seed to make it different from training data.
var testData = mlContext.Data.LoadFromEnumerable(GenerateRandomDataPoints(500, seed:123));

// Run the model on test data set.
var transformedTestData = model.Transform(testData);

// Convert IDataView object to a list.
var predictions = mlContext.Data.CreateEnumerable<Prediction>(transformedTestData, reuseRowObject: false).ToList();

// Look at 5 predictions
foreach (var p in predictions.Take(5))
Console.WriteLine($"Label: {p.Label:F3}, Prediction: {p.Score:F3}");

// Expected output:
// Label: 0.985, Prediction: 0.945
// Label: 0.155, Prediction: 0.104
// Label: 0.515, Prediction: 0.515
// Label: 0.566, Prediction: 0.448
// Label: 0.096, Prediction: 0.082

// Evaluate the overall metrics
var metrics = mlContext.Regression.Evaluate(transformedTestData);
SamplesUtils.ConsoleUtils.PrintMetrics(metrics);

// Expected output:
// Mean Absolute Error: 0.05
// Mean Squared Error: 0.00
// Root Mean Squared Error: 0.06
// RSquared: 0.95
}

private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count, int seed=0)
{
var random = new Random(seed);
float randomFloat() => (float)random.NextDouble();
for (int i = 0; i < count; i++)
{
var label = randomFloat();
yield return new DataPoint
{
Label = label,
// Create random features that are correlated with label.
Features = Enumerable.Repeat(label, 50).Select(x => x + randomFloat()).ToArray()
};
}
}

// Example with label and 50 feature values. A data set is a collection of such examples.
private class DataPoint
{
public float Label { get; set; }
[VectorType(50)]
public float[] Features { get; set; }
}

// Class used to capture predictions.
private class Prediction
{
// Original label.
public float Label { get; set; }
// Predicted score from the trainer.
public float Score { get; set; }
}
}
}
Loading