Provide a clean migration experience from built-in COM to source-generated COM

[jkoritzinsky]

We recently tested out the "convert to source-generated COM experience" on System.Transactions.Local and we found that the existing analyzers/fixers have a few holes that can make adoption more difficult and cause runtime errors.

In particular, it is really easy to end up in a state where some, but not all, interfaces have been (or can be) converted over from built-in COM to source-generated COM.

We feel that this scenario will be decently common due to the following reasons:

• Changing a [ComImport] interface that is part of public surface area to [GeneratedComInterface] is a runtime breaking change.
  • For example, System.Transactions.IDtcTransaction is a public interface, but the rest of the COM surface area in System.Transactions.Local is internal only. We can use Marshal.Get/SetComObjectData to bridge the two systems, but it's ugly and easy to mess up and means that there are two different managed objects to represent the same COM object in the same library.
• Developers may want to use source-generated COM in their code, but they consume COM interface definitions from somewhere else that owns them but has not provided source-generated versions.
  • For example, someone writing a VS extension and using their own COM interfaces and VS's COM interfaces.
• Developers may already be using APIs from the System.Runtime.InteropServices.ComTypes namespace. This namespace has standard COM types that we cannot move to being source-generated due to the prior mentioned breaking change issues.
  • For example, System.Transactions.Local uses IConnectionPointContainer and related connection point sinks, so one of the internal interfaces (and the corresponding implementing C# class) to use built-in COM for that scenario. We can reuse the Marshal.Get/SetComObjectData trick we used above to work around this.

We discussed this issue offline and determined that a multi-pronged approach here would be best. Each of these steps can be done gradually or not at all. They're listed in increasing cost/complexity.

• Update the "using Runtime-implemented COM APIs with source-generated COM types" analyzer to also warn for cases where a source-generated COM type is cast to or from a runtime-implemented COM type. Currently, these cases do not report a diagnostic but will always fail at runtime.
• Provide a runtime configuration option for the following opt-in, not trimming/AOT compliant set of adoption features:
  • At runtime, detect when a source-generated COM object wrapper is cast to a [ComImport] interface, and reflection-emit a type that will enable us to shim the [ComImport] interface to the underlying managed object using the built-in marshalling rules (RCW interop).
  • At compile time, add an implementation of ICustomQueryInterface to all [GeneratedComClass] types that do not already implement the interface, as well as new implementations of. At runtime, this interface implementation will check if the feature flag is enabled, and if so, get a pointer to a compatible interface implementation that does source-generated marshalling for parameters and gets the this managed object from the runtime-based COM interop system.

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

We recently tested out the "convert to source-generated COM experience" on System.Transactions.Local and we found that the existing analyzers/fixers have a few holes that can make adoption more difficult and cause runtime errors.

In particular, it is really easy to end up in a state where some, but not all, interfaces have been (or can be) converted over from built-in COM to source-generated COM.

We feel that this scenario will be decently common due to the following reasons:

• Changing a [ComImport] interface that is part of public surface area to [GeneratedComInterface] is a runtime breaking change.
  • For example, System.Transactions.IDtcTransaction is a public interface, but the rest of the COM surface area in System.Transactions.Local is internal only. We can use Marshal.Get/SetComObjectData to bridge the two systems, but it's ugly and easy to mess up and means that there are two different managed objects to represent the same COM object in the same library.
• Developers may want to use source-generated COM in their code, but they consume COM interface definitions from somewhere else that owns them but has not provided source-generated versions.
  • For example, someone writing a VS extension and using their own COM interfaces and VS's COM interfaces.
• Developers may already be using APIs from the System.Runtime.InteropServices.ComTypes namespace. This namespace has standard COM types that we cannot move to being source-generated due to the prior mentioned breaking change issues.
  • For example, System.Transactions.Local uses IConnectionPointContainer and related connection point sinks, so one of the internal interfaces (and the corresponding implementing C# class) to use built-in COM for that scenario. We can reuse the Marshal.Get/SetComObjectData trick we used above to work around this.

We discussed this issue offline and determined that a multi-pronged approach here would be best. Each of these steps can be done gradually or not at all. They're listed in increasing cost/complexity.

• Update the "using Runtime-implemented COM APIs with source-generated COM types" analyzer to also warn for cases where a source-generated COM type is cast to or from a runtime-implemented COM type. Currently, these cases do not report a diagnostic but will always fail at runtime.
• Provide a runtime configuration option for the following opt-in, not trimming/AOT compliant set of adoption features:
  • At runtime, detect when a source-generated COM object wrapper is cast to a [ComImport] interface, and reflection-emit a type that will enable us to shim the [ComImport] interface to the underlying managed object using the built-in marshalling rules (RCW interop).
  • At compile time, add an implementation of ICustomQueryInterface to all [GeneratedComClass] types that do not already implement the interface, as well as new implementations of. At runtime, this interface implementation will check if the feature flag is enabled, and if so, get a pointer to a compatible interface implementation that does source-generated marshalling for parameters and gets the this managed object from the runtime-based COM interop system.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-System.Runtime.InteropServices, tenet-compatibility
Milestone:	-