Skip to content

Towards #2522 - LinearSvmTrainer documentation #3401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 20, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion src/Microsoft.ML.StandardTrainers/Standard/Online/LinearSvm.cs
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,37 @@
namespace Microsoft.ML.Trainers
{
/// <summary>
/// Linear SVM that implements PEGASOS for training. See: http://ttic.uchicago.edu/~shai/papers/ShalevSiSr07.pdf
/// The <see cref="IEstimator{TTransformer}"/> to predict a target using a linear binary classification model
/// trained with Linear SVM.
/// </summary>
/// <remarks>
/// <format type="text/markdown"><![CDATA[
/// To create this trainer, use [LinearSvm](xref:Microsoft.ML.StandardTrainersCatalog.LinearSvm(Microsoft.ML.BinaryClassificationCatalog.BinaryClassificationTrainers,System.String,System.String,System.String,System.Int32))
/// or [LinearSvm(Options)](xref:Microsoft.ML.StandardTrainersCatalog.LinearSvm(Microsoft.ML.BinaryClassificationCatalog.BinaryClassificationTrainers,Microsoft.ML.Trainers.LinearSvmTrainer.Options)).
///
/// [!include[io](~/../docs/samples/docs/api-reference/io-columns-binary-classification.md)]
///
/// ### Trainer Characteristics
/// | | |
/// | -- | -- |
/// | Machine learning task | Binary classification |
/// | Is normalization required? | Yes |
/// | Is caching required? | No |
/// | Required NuGet in addition to Microsoft.ML | None |
///
/// ### Training Algorithm Details
/// Linear [SVM](https://en.wikipedia.org/wiki/Support-vector_machine#Linear_SVM) is a trainer that implements
/// an algorithm that finds a hyperplane in the feature space for binary classification, by solving an [SVM problem](https://en.wikipedia.org/wiki/Support-vector_machine#Computing_the_SVM_classifier).
/// For instance, with feature values $f_0, f_1,..., f_{D-1}$, the prediction is given by determining what side of the hyperplane the point falls into.
/// That is the same as the sign of the feautures' weighted sum, i.e. $\sum_{i = 0}^{D-1} \left(w_i * f_i \right)$, where $w_0, w_1,..., w_{D-1}$ are the weights computed by the algorithm.
///
/// This algorithm uses the PEGASOS method, which alternates between stochastic gradient descent steps and projection steps,
/// introduced in [this paper](http://ttic.uchicago.edu/~shai/papers/ShalevSiSr07.pdf) by Shalev-Shwartz, Singer and Srebro.
/// ]]>
/// </format>
/// </remarks>
/// <seealso cref="StandardTrainersCatalog.LinearSvm(BinaryClassificationCatalog.BinaryClassificationTrainers, LinearSvmTrainer.Options)"/>
/// <seealso cref="StandardTrainersCatalog.LinearSvm(BinaryClassificationCatalog.BinaryClassificationTrainers, string, string, string, int)"/>
public sealed class LinearSvmTrainer : OnlineLinearTrainer<BinaryPredictionTransformer<LinearBinaryModelParameters>, LinearBinaryModelParameters>
{
internal const string LoadNameValue = "LinearSVM";
Expand All @@ -40,6 +69,10 @@ public sealed class LinearSvmTrainer : OnlineLinearTrainer<BinaryPredictionTrans

internal readonly Options Opts;

/// <summary>
/// Options for the <see cref="LinearSvmTrainer"/> as used in
/// <see cref="StandardTrainersCatalog.LinearSvm(BinaryClassificationCatalog.BinaryClassificationTrainers, Options)"/>.
/// </summary>
public sealed class Options : OnlineLinearOptions
{
[Argument(ArgumentType.AtMostOnce, HelpText = "Regularizer constant", ShortName = "lambda", SortOrder = 50)]
Expand Down
38 changes: 9 additions & 29 deletions src/Microsoft.ML.StandardTrainers/StandardTrainersCatalog.cs
Original file line number Diff line number Diff line change
Expand Up @@ -800,22 +800,12 @@ public static PairwiseCouplingTrainer PairwiseCoupling<TModel>(this MulticlassCl
}

/// <summary>
/// Predict a target using a linear binary classification model trained with the <see cref="LinearSvmTrainer"/> trainer.
/// Create a <see cref="LinearSvmTrainer"/>, which predicts a target using a linear binary classification model trained
/// over boolean label data.
/// </summary>
/// <remarks>
/// <para>
/// The idea behind support vector machines (SVM), is to map instances into a high dimensional space
/// in which the two classes are linearly separable, i.e., there exists a hyperplane such that all the positive examples are on one side of it,
/// and all the negative examples are on the other.
/// </para>
/// <para>
/// After this mapping, quadratic programming is used to find the separating hyperplane that maximizes the
/// margin, i.e., the minimal distance between it and the instances.
/// </para>
/// </remarks>
/// <param name="catalog">The <see cref="BinaryClassificationCatalog"/>.</param>
/// <param name="labelColumnName">The name of the label column. </param>
/// <param name="featureColumnName">The name of the feature column.</param>
/// <param name="catalog">The binary classification catalog trainer object.</param>
/// <param name="labelColumnName">The name of the label column. The column data must be <see cref="System.Boolean"/>.</param>
/// <param name="featureColumnName">The name of the feature column. The column data must be a known-sized vector of <see cref="System.Single"/>.</param>
/// <param name="exampleWeightColumnName">The name of the example weight column (optional).</param>
/// <param name="numberOfIterations">The number of training iteraitons.</param>
public static LinearSvmTrainer LinearSvm(this BinaryClassificationCatalog.BinaryClassificationTrainers catalog,
Expand All @@ -829,21 +819,11 @@ public static LinearSvmTrainer LinearSvm(this BinaryClassificationCatalog.Binary
}

/// <summary>
/// Predict a target using a linear binary classification model trained with the <see cref="LinearSvmTrainer"/> trainer.
/// Create a <see cref="LinearSvmTrainer"/> with advanced options, which predicts a target using a linear binary classification model
/// trained over boolean label data.
/// </summary>
/// <remarks>
/// <para>
/// The idea behind support vector machines (SVM), is to map instances into a high dimensional space
/// in which the two classes are linearly separable, i.e., there exists a hyperplane such that all the positive examples are on one side of it,
/// and all the negative examples are on the other.
/// </para>
/// <para>
/// After this mapping, quadratic programming is used to find the separating hyperplane that maximizes the
/// margin, i.e., the minimal distance between it and the instances.
/// </para>
/// </remarks>
/// <param name="catalog">The <see cref="BinaryClassificationCatalog"/>.</param>
/// <param name="options">Advanced arguments to the algorithm.</param>
/// <param name="catalog">The binary classification catalog trainer object.</param>
/// <param name="options">Trainer options.</param>
public static LinearSvmTrainer LinearSvm(this BinaryClassificationCatalog.BinaryClassificationTrainers catalog,
LinearSvmTrainer.Options options)
{
Expand Down