Change the way we build CoreCLR tests for NativeAOT

[MichalStrehovsky]

This is two commits: the first one basically just rolls back #62704 - the original strategy for building CoreCLR tests with NativeAOT. In this mode, the test build would generate project files that the test execution script would MSBuild and then execute the result. We did it that way because AOT compiling tests takes time and it's better if we can fan in out into Helix. The main disadvantage is that this strategy requires the target Helix machine to have the necessary tools to finish the compilation (e.g. we would need a Windows SDK and VS to be installed on the ARM64 Helix machines since the AOT compilation requires those).

The need to fan out compilation will greatly disappear once we don't have 6000 tiny tests to compile. I also find it unlikely that we would run the whole Pri-0 suite as part of the CI. Most of these tests are codegen tests and there's very few NativeAOT specific codegen bugs.

So in the second commit, I'm adding a different NativeAOT test strategy - we NativeAOT compile the test as part of the test build. To run the test, we use the custom launcher infrastructure to execute the AOT compiled test instead of using corerun on a managed assembly.

[EgorBo]

I'd be nice if when I run all tests using src/tests/run.cmd it could print commands for ILC into output, so in case of a failure/assert I could just copy-paste them and run in ilc.sln

[MichalStrehovsky]

Right now (before this pull request is merged), you can set CLRTestNoCleanup=1 and it will leave around the RSP files for both ILC and link.exe.

This pull request doesn't do the deletion of artifact because it's weird to do that as part of the build (and breaks incrementality). I'm hoping that nobody will try to build a full Pri-0 test pass while low on disk space because it takes 100+ GB to do that. Since artifacts are not deleted, the response files will be left intact.

I realized I'll also have to update the docs.

[jkoritzinsky]

It looks like this PR removes support in our testing scripts for the NativeAOT multi-module testing mode. Is that intentional?

[MichalStrehovsky]

It looks like this PR removes support in our testing scripts for the NativeAOT multi-module testing mode. Is that intentional?

I wanted to scope this change down because less functionality can be reviewed more easily, but on a second thought this is probably just a dozen extra lines.

[MichalStrehovsky]

The mild inconvenience for multi-obj-file-compilation testing is that we now have to decide how we build the test at the time of test build instead of at the time of test execution. In the old setup, we would simply re-run the tests twice - once with single obj compilation, and the other time with multi-obj compilation.

Now we would basically need a new test leg and I definitely don't want that for each pull request.

Here's how I'm solving it.

• If the test build is executed with /p:BuildNativeAotFrameworkObjects=true, we'll precompile the framework into a single single object archive, but won't actually build the tests in this mode. CI will pass this parameter.
• If the test build is executed with /p:IlcMultiModule=true, we'll precompile framework and compile all the tests in multifile mode. We might want to use this for Pri-0 test passes at some point.
• I'm adding an old new test from the CoreRT repo that is always forced into multimodule compilation. We skip it if the framework wasn't prebuilt. That way we don't lose coverage of multifile and keep most of the benefits of this coverage in the CI. I'm making it skippable because precompiling the framework takes minutes and it's annoying locally.

[MichalStrehovsky]

@jkoritzinsky @trylek I think this is ready - could you have a look?

It's two commits - the first is basically a rollback.

The second is the new testing strategy.

I'm not actually hooking this into helix yet because this change is already too big but I think it should be straightforward.

@jkotas could you have a look at the changes under src/coreclr/nativeaot? I had to add a couple extensibility points that hopefully make sense.

[jkotas]

@jkotas could you have a look at the changes under src/coreclr/nativeaot?

LGTM

[trylek]

LGTM, thank you!

[trylek]

These synthesized property values differ from similar definitions in Microsoft.NETCore.Native.Unix.props (by the extra /.. parent folder level), is that intentional?

[MichalStrehovsky]

Yes, I tried to capture it in the comment above this line - the test build places the NativeAOT compiled artifacts in a native subdirectory, but we might have .so/.dll files that the test depends on one level up. We tweak rpath so that it loads from there. On Windows, we search the current directory, so it seems to work fine.

[trylek]

Non-blocking - I wonder whether this logic should be rather put in one of the common scripts like Directory.Build.targets so that it doesn't need to be open-coded in the individual tests. Or is this special to just this one test?

[MichalStrehovsky]

It's specific to the MultiModule compilation mode that we don't ship... it should eventually be part of the product. We don't need it for other tests.