Remove Google.Protobuf dependency from Microsoft.ML

[eerhardt]

System information

• OS version/distro: all
• .NET Version (eg., dotnet --info): all

Issue

• What did you do?
  Install the base Microsoft.ML nuget package

• What happened?
  It also adds a reference to Google.Protobuf

• What did you expect?
  I expected that the base Microsoft.ML nuget package would have minimal dependencies. From my understanding, Google.Protobuf is only used in ML.NET for the ONNX support, which isn't needed all of the time. We should move the ONNX support into a separate NuGet package that developers can opt into. That way developers who aren't using this functionality won't need to pay for this dependency.

Source code / logs

machinelearning/src/Microsoft.ML.UniversalModelFormat/Microsoft.ML.UniversalModelFormat.csproj

Line 9 in 436700a

<PackageReference Include="Google.Protobuf" Version="$(GoogleProtobufPackageVersion)" />

is the only place that uses this dependency. This project should be factored out of the base package, similar to what we've done with the Microsoft.ML.Parquet package.

/cc @glebuk @TomFinley

[shauheen]

The protobuf dependency is for ONNX export. We need this as the feature becomes available.

[TomFinley]

Technical difficulties aside for the moment, I'd like to understand what policy we're proposing to adopt. Certainly up till now the criteria for taking a nuget dependency has never previously been that it is "needed all of the time," nor would I agree that's a reasonable policy since as far as I see nothing here is needed all of the time.

[eerhardt]

Certainly up till now the criteria for taking a nuget dependency has never previously been that it is "needed all of the time,"

I don't think there has been a previous criteria.

@glebuk and I met a few weeks ago, and here was our rough sketch for what the NuGet layout should look like:

ML.NET Nuget Packages: 
- ML.NET.Core - IDV + c#/only components + cpumath, maybe MKL, NewtonSoft 

  Additional learners: 
    <- ML.NET.LighGBM (LightGBM + unmanaged dependencies for LighGbm) 
    <- ML.NET.Cntk (CNTK wrapper for ML.NET) 

  Distributed 
    <- ML.NET.Parquet - Parquet loader for distributed ML.NET

  Additional Transforms 
    <- ML.NET.Text - transforms 
      - Also depends on ML.NET.Text.Resources.* resources for text 

    <- ML.NET.Image - Image tranforms 
      <- ML.NET.Image.Models Resnet18, 50, 101, Alexnet 
        <- ML.NET.Image - OpenCV, MKL etc - image transforms 

  Inference? 
    <- ML.NET.Onnx - ONNX converters, utils 

The idea being that a user opts in to functionality and dependencies as they need it. For example, that way someone who isn't using CNTK or LighGBM doesn't need to take dependencies on them.

A good starting policy I'd propose would be something along the lines of "If the majority of ML.NET users aren't going to use a set of functionality, it shouldn't be in the core NuGet package. Especially if there are external dependencies or size considerations when including the functionality."

[TomFinley]

The situation is very, very different from the Parquet loader or the CNTK learner wrapper, from a technical perspective. The Parquet loader is a specific IDataLoader implementation, but no existing core component's behavior changes due to its presence. It is completely isolated. On the other hand, ONNX exporting, like all exporting in this codebase, is done by having the component (either an IDataTransform or IPredictor) implement an interface, i.e., ICanSaveOnnx or one of the descending more specific interfaces.

[markusweimer]

TomFinley, would it be possible to externalize the export functionality from the classes? That way, the ONNX export could be its own NuGet.

[TomFinley]

Yes, abstracting away the classes the components actually interact with behind interfaces will allow us to do this. I have published a PR linked above.