Expose all LightGbm Metrics

[superichmann]

Is your feature request related to a problem? Please describe.
no

Describe the solution you'd like
Add all metrics as stated here

Describe alternatives you've considered
migrate to python

Additional context
Add any other context or screenshots about the feature request here.

[michaelgsharp]

I don't think will want to expose them directly in the Options, but we can probably do this using the new advanced options stuff we have pretty easily.

@luisquintanilla @ericstj thoughts?

[ericstj]

Can you share what this would look like and describe what changes we'd need to make, @michaelgsharp?

[michaelgsharp]

So we have an options class that has how we expose it in C# that we would have to add them all to that looks like this

/// <summary>
/// Whether to force column-wise histogram building.
/// </summary>
[Argument(ArgumentType.AtMostOnce, HelpText = "Whether to force column-wise histogram building.")]
public bool ForceColumnWise = false;

We would then have to add them to our mapping dictionary that maps our name to the light gbm name

{nameof(ForceColumnWise),                      "force_col_wise"},

and add it to where we create the options dictionary we pass to lightgbm

res[GetOptionName(nameof(ForceRowWise))] = ForceRowWise;

I think those 3 steps pretty much cover it.

There are about 60 LightGBM training options and we currently expose about 1/5 of them. So its probably only a couple hundred extra lines. Not a lot of work or even hard to do, would just add a lot more options people are exposed to.

I think this is where that new advanced options stuff would fit perfectly. Make it so that people can set whatever lightGBM options there they want and have them passed in. That way power users that want those options can still pass them in, but normal users don't have to think about them. Unless we think normal users will want all those options too.

[ericstj]

I think this is where that new advanced options stuff would fit perfectly. Make it so that people can set whatever lightGBM options there they want and have them passed in. That way power users that want those options can still pass them in, but normal users don't have to think about them. Unless we think normal users will want all those options too.

This was what I was hoping to outline, not the existing mechanism. Maybe we document something like "LightGbmAdditionalOptions" and let folks set this as Dictionary<string, object> then we just fetch that and join it with other strongly-typed options, allowing those to override?

[michaelgsharp]

@ericstj
So in the MLContext we can add custom options https://github.com/dotnet/machinelearning/blob/main/src/Microsoft.ML.Data/MLContext.cs#L202.

I was thinking we could make a dictionary of light gbm options and then add it to the options.

Dictionary<string, object> lgbmOptions = new Dictionary<string, object>();
// Add whatever options we want to the dictionary then add that to the context options.
...
mlcontext.TryAddOptions("lgbmOptions", lgbmOptions);

Then inside the lightgbm trainer where it is creating the options normally we can just add the custom options. One place its done is here, https://github.com/dotnet/machinelearning/blob/main/src/Microsoft.ML.LightGbm/LightGbmTrainerBase.cs#L281, though there may be a better place.

if (((IHostEnvironmentInternal)host).TryGetOption<Dictionary<string, object>>("lgbmOptions", out Dictionary<string, object> lbgmOptions")
{
    // add all of lgbmOptions dictionary to res dictionary
}

obviously we need error handling, and probably an enum/const or something so the options name is not free form, but this is the general approach I would take.