[Umbrella] Refresh IL verification

[jcouv]

Recent versions of the C# compiler produce unverifiable code (by current spec). All new compiler tests that hit this just disable the verification step, but we'd like to reverse the trend.
Below is a rough plan how we can be back to verifying the compiler output.

• Automate compiling IL tests and running those tests in CI (Enable building and running ILVerification tests by default corert#6307, thanks Jan!)
• Switch to ILVerify in compiler test suite
  • Need fully-signed ILVerify
  • Solve target framework issue (ILVerify targets netcoreapp2.0, but compiler emit tests target net46)
  • Work around the signing/strong name issue for local testing
  • Hack a primitive integration of Roslyn tests with ILVerify
  • Get all Roslyn tests to pass
    • Triage remaining observable differences between PEVerify and ILVerify (done at https://github.com/dotnet/corert/issues/4958)
    • Performance seems fine
    • Add support for multi-module assemblies (dotnet/corert@eec7036)
  • Tighten Roslyn tests for expected verification error (currently we skip verification, but should verify and check fails as expected, see Tighten PE verification in CompileAndVerify #23132)
  • Tighten Roslyn tests for unsafe code (many of them should still be verifiable, but currently skip verification, see Enable PE verification in tests of unsafe code #23130)
  • Need package (Andon) (published at https://dotnet.myget.org/feed/dotnet-core/package/nuget/Microsoft.DotNet.ILVerification )
  • Need APIs (see [ILVerify] Define and Implement a public API surface corert#3734, PR Add public API for ILVerify corert#5186)
  • Split into exe+dll (PR ILVerify: split into exe and library corert#5258)
  • Review verification rules that are not yet implemented and flag which are critical for us (done at: https://github.com/dotnet/corert/issues/4958)
• Add support for ref return
  • Draft informal spec
  • Review proposal (with Xamarin, Jan/Michal, compiler)
  • Update compiler (separate slots by returnability and tag slots per proposal)
  • Update ILVerify
  • Formalize spec
• Add support for readonliness (after ref return is done)
  • Draft informal spec
• Add support for Span (after ref return is done)
• Add support for runtime-async methods ILVerify support for runtime async runtime#118042
• Spec bug around delegate assignment

Tagging @VSadov @gafter and folks who have been contributing to ILVerify (with our thanks): @ArztSamuel @gregkalapos @mmayr-at @discostu105

[jcouv]

@VSadov posted a proposal for informal spec for ref return: dotnet/designs#17

[jcouv]

@VSadov Note that ILVerify is currently only built for target framework netcoreapp2.0, but the compiler emit tests aren't yet built or run for this target. We'll either have to change ILVerify to also target net46, or we'll have to find a way for emit tests to target netcoreapp2.0...

[ArztSamuel]

ILVerify is now also targeting net46, since dotnet/corert#4862.

[VSadov]

FYI: updated with the reference to another draft. Now for the "readonly references" rules.

dotnet/designs#21

[jcouv]

I went through the C# emit tests and classified the observable differences in behavior between ILVerify and PEVerify. Also updated the OP notes.

I need to investigate some of them to confirm whether they are a Roslyn problem or an ILVerify problem (for instance, I suspect the one labelled "Mscorlib" may be a Roslyn issue). Also some issues are not high-priority (the one labelled "NoPIA").

The ones I'm pretty sure about: ILVerify doesn't support net modules (only supports assemblies), and it doesn't support TypedReference. I'll file bugs and then we can discuss which issues are more important.

[VSadov]

I have tried to order remaining unimplemented error IDs from https://github.com/dotnet/corert/blob/master/src/ILVerify/src/VerifierError.cs

I do not have a lot of insight if some are easier to implement than others within existing infrastructure. I just looked at possibility to see particular violations as a result of some compiler bug.
In that sense some errors could be more "useful" than others.

I have listed "interesting" errors here: https://github.com/dotnet/corert/issues/4958