Skip to content

Add support for NativeAOT testing #62704

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 14, 2021
Merged

Add support for NativeAOT testing #62704

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1af4682
Add support for NativeAOT testing
MichalStrehovsky Dec 13, 2021
1f114f0
Work around batch oddness
MichalStrehovsky Dec 14, 2021
8b670a6
Clean up artifacts on Unix too
MichalStrehovsky Dec 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions src/tests/Common/CLRTest.Execute.Bash.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,10 @@ then
else
LAUNCHER="$_DebuggerFullPath $_DebuggerArgsSeparator $(CLRTestRunFile)"
fi
if [ ! -z "$RunNativeAot" ]
then
LAUNCHER="$_DebuggerFullPath"
fi

$(BashIlrtTestLaunchCmds)

Expand All @@ -336,6 +340,13 @@ CLRTestExitCode=$?
if [ ! -z ${RunCrossGen+x} ]%3B then
ReleaseLock
fi
if [ ! -z "$RunNativeAot" ]
then
if [ -z "$CLRTestNoCleanup" ];
then
rm -rf native
fi
fi

$(BashLinkerTestCleanupCmds)
]]></BashCLRTestLaunchCmds>
Expand Down
8 changes: 8 additions & 0 deletions src/tests/Common/CLRTest.Execute.Batch.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,9 @@ IF NOT "%CLRCustomTestLauncher%"=="" (
) ELSE (
set LAUNCHER=%_DebuggerFullPath% $(CLRTestRunFile)
)
IF NOT "%RunNativeAot%"=="" (
set LAUNCHER=%_DebuggerFullPath%
)

$(BatchIlrtTestLaunchCmds)

Expand All @@ -318,6 +321,11 @@ set CLRTestExitCode=!ERRORLEVEL!
if defined RunCrossGen (
call :ReleaseLock
)
if defined RunNativeAot (
if not defined CLRTestNoCleanup (
IF EXIST native rmdir /s /q native
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this just in the Windows version of the script and how does it play together with similar clauses in the execution script snippet generator in CLRTest.NativeAOT.targets?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good catch. I guess we never run enough tests on Unix systems where not deleting the results would be a problem. I'll fix that.

)
)
$(BatchLinkerTestCleanupCmds)
]]></BatchCLRTestLaunchCmds>
<BatchCLRTestLaunchCmds Condition="'$(CLRTestKind)' == 'BuildAndRun' And $(TargetOS) == 'Android' ">
Expand Down
1 change: 1 addition & 0 deletions src/tests/Common/CLRTest.Execute.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ This file contains the logic for providing Execution Script generation.

<Import Project="CLRTest.Jit.targets" />
<Import Project="CLRTest.CrossGen.targets" />
<Import Project="CLRTest.NativeAot.targets" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit - we may want to check somewhere that RunCrossgen2 and RunNativeAOT aren't set at the same time, otherwise I suspect interesting things may happen.

<Import Project="CLRTest.GC.targets" />
<Import Project="CLRTest.Execute.*.targets" />
<Import Project="CLRTest.MockHosting.targets" Condition="'$(RequiresMockHostPolicy)' == 'true'" />
Expand Down
190 changes: 190 additions & 0 deletions src/tests/Common/CLRTest.NativeAot.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
<!--
***********************************************************************************************
CLRTest.Execute.targets

WARNING: DO NOT MODIFY this file unless you are knowledgeable about MSBuild and have
created a backup copy. Incorrect changes to this file will make it
impossible to load or build your projects from the command-line or the IDE.

This file contains the logic for providing Execution Script generation.

WARNING: When setting properties based on their current state (for example:
<Foo Condition="'$(Foo)'==''>Bar</Foo>). Be very careful. Another script generation
target might be trying to do the same thing. It's better to avoid this by instead setting a new property.

Additionally, be careful with itemgroups. Include will propagate outside of the target too!

***********************************************************************************************
-->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<BashScriptSnippetGen>$(BashScriptSnippetGen);GetNativeAotBashScript</BashScriptSnippetGen>
<BatchScriptSnippetGen>$(BatchScriptSnippetGen);GetNativeAotBatchScript</BatchScriptSnippetGen>
</PropertyGroup>

<!--
This returns the portion of the execution script that generates the required lines to run the test with native AOT.

Tests can opt out of ahead-of-time (AOT) compilation by setting this property in their project:

<NativeAotTest>false</NativeAotTest>
-->
<Target Name="GetNativeAotBashScript">

<PropertyGroup>
<NativeAotCleanupBashScript Condition="'$(CLRTestKind)' == 'BuildAndRun' and '$(NativeAotTest)' != 'false'">
<![CDATA[
if [ ! -z ${RunNativeAot+x} ]%3B then
rm -rf native
fi
]]>
</NativeAotCleanupBashScript>

<NativeAotBashScript Condition="'$(CLRTestKind)' == 'BuildAndRun' and '$(NativeAotTest)' != 'false'">
<![CDATA[
if [ ! -z ${RunNativeAot+x} ]%3B then
__Command=$_DebuggerFullPath
# Tests run locally need __TestDotNetCmd (set by runtest.py) or a compatible 5.0 dotnet runtime in the path
if [ ! -z ${__TestDotNetCmd+x} ] %3B then
__Command+=" $__TestDotNetCmd"
else
__Command+=" dotnet"
fi

__Command+=" msbuild /p:IlcPath=$CORE_ROOT/nativeaot nativebuild.proj"
echo "Running Native AOT: $__Command"
$__Command
__naotExitCode=$?

if [ $__naotExitCode -ne 0 ]; then
echo Native AOT failed with exitcode: $__naotExitCode
exit 1
fi

# Copy the native component to the directory where the executable resides.
if [[ "$OSTYPE" == "darwin"* ]]; then
cp *.dylib native/ 2>/dev/null
else
cp *.so native/ 2>/dev/null
fi

ExePath=native/$(MSBuildProjectName)
fi
]]>
</NativeAotBashScript>

<BashCLRTestPreCommands>$(NativeAotCleanupBashScript);$(BashCLRTestPreCommands);$(NativeAotBashScript)</BashCLRTestPreCommands>
</PropertyGroup>
</Target>

<Target Name="GetNativeAotBatchScript">
<PropertyGroup>
<NativeAotMultimoduleIncompatibleScript Condition="'$(NativeAotMultimoduleIncompatible)' == 'true'">
<![CDATA[
ECHO SKIPPING EXECUTION BECAUSE TEST IS NOT COMPATIBLE WITH MULTIMODULE COMPILATION
popd
Exit /b 0
]]>
</NativeAotMultimoduleIncompatibleScript>

<NativeAotCleanupBatchScript Condition="'$(CLRTestKind)' == 'BuildAndRun' and '$(NativeAotTest)' != 'false'">
<![CDATA[
if defined RunNativeAot (
rem Delete any leftover native compilation bits
IF EXIST native rmdir /s /q native
)
]]>
</NativeAotCleanupBatchScript>

<NativeAotBatchScript Condition="'$(CLRTestKind)' == 'BuildAndRun' and '$(NativeAotTest)' != 'false'">
<![CDATA[
if defined RunNativeAot (

set ExePath=native\$(MSBuildProjectName)

set __Command=!_DebuggerFullPath!
REM Tests run locally need __TestDotNetCmd (set by runtest.py) or a compatible 5.0 dotnet runtime in the path
if defined __TestDotNetCmd (
set __Command=!__Command! "!__TestDotNetCmd!"
) else (
set __Command=!__Command! "dotnet"
)

set __Command=!__Command! msbuild /p:IlcPath=!CORE_ROOT!\nativeaot nativebuild.proj

if defined NativeAotMultimodule (
set __Command=!__Command! /p:IlcMultiModule=true /p:DisableFrameworkLibGeneration=true /p:FrameworkLibPath=!CORE_ROOT!\nativeaot\sdk
$(NativeAotMultimoduleIncompatibleScript)
)

echo "!__Command!"
call !__Command!

if errorlevel 1 (
ECHO END COMPILATION - FAILED
ECHO FAILED
exit /b 1
)
)
]]>
</NativeAotBatchScript>

<CLRTestBatchPreCommands>$(NativeAotCleanupBatchScript);$(CLRTestBatchPreCommands);$(NativeAotBatchScript)</CLRTestBatchPreCommands>
</PropertyGroup>
</Target>

<Target Name="CreateNativeAotBuildProjectFile"
Condition="'$(_WillCLRTestProjectBuild)' == 'true'"
Inputs="$(MSBuildProjectFullPath)"
Outputs="$(OutputPath)\nativebuild.proj"
BeforeTargets="Build">
<PropertyGroup>
<_RootEntryAssemblyLine Condition="$(CLRTestFullTrimming) != 'true'">&lt;TrimmerDefaultAction&gt;copyused&lt;/TrimmerDefaultAction&gt;</_RootEntryAssemblyLine>

<_NativeAotBuildProjectFile>
<![CDATA[
<Project DefaultTargets="LinkNative" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<PropertyGroup>
<TargetName>$(MSBuildProjectName)</TargetName>
<TargetExt>.dll</TargetExt>
<OutputType>Exe</OutputType>
<OutputPath>%24(MSBuildProjectDirectory)\</OutputPath>
<IntermediateOutputPath>%24(MSBuildProjectDirectory)\</IntermediateOutputPath>
<TargetArchitecture>$(TargetArchitecture)</TargetArchitecture>
<Optimize>$(Optimize)</Optimize>
<DebugSymbols>true</DebugSymbols>
<NETCoreSdkVersion>$(NETCoreSdkVersion)</NETCoreSdkVersion>
$(_RootEntryAssemblyLine)
</PropertyGroup>

$(NativeAotProjectLines)

<ItemGroup>
<!-- Some tests consist of multiple assemblies - make sure ILC sees them -->
<IlcCompileInput Include="%24(MSBuildProjectDirectory)\*.dll" Exclude="%24(MSBuildProjectDirectory)\$(MSBuildProjectName).dll" />

<!-- Copy RuntimeHostConfigurationOptions -->
@(RuntimeHostConfigurationOption->' <RuntimeHostConfigurationOption Include="%(Identity)" Value="%(Value)" />', '%0A')
</ItemGroup>

<!-- We don't do anything in the Compile step since the test is already compiled to IL - the AOT compiler hooks up after this -->
<Target Name="Compile" />

<Target Name="PrepareForILLink" />

<Import Project="%24(IlcPath)\build\Microsoft.NETCore.Native.targets" />

</Project>
]]>
</_NativeAotBuildProjectFile>
</PropertyGroup>

<!-- Write the file -->
<WriteLinesToFile
File="$(OutputPath)\nativebuild.proj"
Lines="$(_NativeAotBuildProjectFile)"
Overwrite="true" />
</Target>

</Project>
26 changes: 25 additions & 1 deletion src/tests/Common/Directory.Build.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
DependsOnTargets="ResolveAssemblyReferences;ResolveRuntimeFilesFromLocalBuild">

<ItemGroup>
<RunTimeDependencyExclude Include="$(CORE_ROOT)\**\*.*" />
<RunTimeDependencyExclude Include="$(CORE_ROOT)\**\*.*" Exclude="$(CORE_ROOT)\nativeaot\**\*.*" />
<RunTimeDependencyExcludeFiles Include="@(RunTimeDependencyExclude -> '%(FileName)%(Extension)')" />
<RunTimeDependencyExcludeFiles Include="@(RunTimeDependencyExclude -> '%(FileName).ni%(Extension)')" />
<RunTimeDependencyExcludeFiles Include="@(RunTimeDependencyExclude -> '%(FileName).pdb')" />
Expand Down Expand Up @@ -171,6 +171,19 @@
<RunTimeDependencyCopyLocal Include="$(CoreDisToolsLibrary)" Condition="$(CopyCoreDisToolsToCoreRoot)" />
</ItemGroup>

<ItemGroup Condition="'$(TestBuildMode)' == 'nativeaot'">
<AotCompilerCopyLocal Include="$(CoreCLRArtifactsPath)ilc/**/*" TargetDir="nativeaot/tools/" />
<AotCompilerCopyLocal Include="$(CoreCLRArtifactsPath)aotsdk/**/*" TargetDir="nativeaot/sdk/" />
<AotCompilerCopyLocal Include="$(CoreCLRArtifactsPath)build/**/*" TargetDir="nativeaot/build/" />
<AotCompilerCopyLocal Include="$(TargetingPackPath)/*" TargetDir="nativeaot/framework/" />

<!-- Works around https://github.com/dotnet/runtime/issues/62372 -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue was closed about a week ago or so, do we still need this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way the issue got resolved is very much not ideal (it placed a similar filter into mono's AOT step, so this filter is also needed).

Correct fix should have been to make it so that the problematic DLL doesn't get into CORE_ROOT in the first place. The fact we still have the file in CORE_ROOT means we're sending a garbage file to all test machines.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I see, that's unfortunate indeed, I assume that the right fix would be to adjust the filtering clauses around

runtime/src/tests/Directory.Build.targets

Line 294 in 03194f7

<ItemGroup>

I defer to Jeremy whether we should reopen the issue as not fully resolved or open another issue for the proper solution, link to a closed issue in a code comment is typically interpreted as 'obsolete link to an issue that has since been fixed' and after some time it may become quite a pain to track down what the original problem was and what is its current status.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we should make sure that DllImportGenerator doesn’t get put into Core_Root in the first place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll look into the DllImportGenerator issue in a separate pull request because I don't want this pull request to be a functional change.

<_FixedRuntimeCopyLocalItems Include="@(RuntimeCopyLocalItems)" />
<_FixedRuntimeCopyLocalItems Remove="@(_FixedRuntimeCopyLocalItems)" Condition="'%(Filename)' == 'Microsoft.Interop.DllImportGenerator'" />

<AotCompilerCopyLocal Include="@(_FixedRuntimeCopyLocalItems)" TargetDir="nativeaot/framework/" />
</ItemGroup>

<ItemGroup Condition="'$(RuntimeFlavor)' == 'mono' and '$(IsDesktopOS)' == 'true' " >
<RuntimeDependencyCopyLocal Include="$(MonoArtifactsPath)/libcoreclr$(LibSuffix)" TargetDir="" />
<RuntimeDependencyCopyLocal Include="$(MonoArtifactsPath)/libmono-component-*" TargetDir="" />
Expand All @@ -188,6 +201,17 @@
<Output TaskParameter="DestinationFiles" ItemName="FileWrites" />
</Copy>

<Copy
SourceFiles="@(AotCompilerCopyLocal)"
DestinationFiles="@(AotCompilerCopyLocal -> '$(CORE_ROOT)/%(TargetDir)%(RecursiveDir)%(Filename)%(Extension)')"
SkipUnchangedFiles="$(SkipCopyUnchangedFiles)"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Retries="$(CopyRetryCount)"
RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
UseHardlinksIfPossible="$(CreateHardLinksForCopyFilesToOutputDirectoryIfPossible)">
<Output TaskParameter="DestinationFiles" ItemName="FileWrites" />
</Copy>

</Target>

</Project>
1 change: 1 addition & 0 deletions src/tests/build.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ if /i "%1" == "buildagainstpackages" (echo error: Remove /BuildAgainstPackages
if /i "%1" == "crossgen2" (set __TestBuildMode=crossgen2&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
if /i "%1" == "composite" (set __CompositeBuildMode=1&set __TestBuildMode=crossgen2&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
if /i "%1" == "pdb" (set __CreatePdb=1&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
if /i "%1" == "nativeaot" (set __TestBuildMode=nativeaot&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
if /i "%1" == "perfmap" (set __CreatePerfmap=1&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
if /i "%1" == "Exclude" (set __Exclude=%2&set processedArgs=!processedArgs! %1 %2&shift&shift&goto Arg_Loop)
if /i "%1" == "-priority" (set __Priority=%2&shift&set processedArgs=!processedArgs! %1=%2&shift&goto Arg_Loop)
Expand Down
11 changes: 11 additions & 0 deletions src/tests/build.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -552,4 +552,15 @@
DependsOnTargets="MonoAotCompileTests"
AfterTargets="ManagedBuild"
Condition="'$(__BuildTestWrappersOnly)' != '1' and '$(__GenerateLayoutOnly)' != '1' and '$(__CopyNativeTestBinaries)' != '1' and ($(MonoAot) or $(MonoFullAot))" />

<Target Name="NativeAotCompileFramework" AfterTargets="CopyDependencyToCoreRoot" Condition="'$(NativeAotMultimodule)' == 'true'">
<!-- MSBuild doesn't understand the Checked configuration -->
<PropertyGroup>
<_OptimizeFlagValue Condition="'$(Configuration)' == 'Checked' or '$(Configuration)' == 'Release'">true</_OptimizeFlagValue>
</PropertyGroup>
<MSBuild Projects="$(CORE_ROOT)/nativeaot/build/BuildFrameworkNativeObjects.proj"
Targets="CreateLib"
Properties="FrameworkLibPath=$(CORE_ROOT)/nativeaot/sdk;FrameworkObjPath=$(BaseIntermediateOutputPath)/NativeAOTFX;IlcPath=$(CORE_ROOT)/nativeaot;Optimize=$(_OptimizeFlagValue);NETCoreSdkVersion=6.0.0" />
</Target>

</Project>
5 changes: 5 additions & 0 deletions src/tests/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ usage_list+=("-dir:xxx - build all tests in a given directory");
usage_list+=("-tree:xxx - build all tests in a given subtree");

usage_list+=("-crossgen2: Precompiles the framework managed assemblies in coreroot using the Crossgen2 compiler.")
usage_list+=("-nativeaot: Builds the tests for Native AOT compilation.")
usage_list+=("-priority1: include priority=1 tests in the build.")
usage_list+=("-composite: Use Crossgen2 composite mode (all framework gets compiled into a single native R2R library).")
usage_list+=("-perfmap: emit perfmap symbol files when compiling the framework assemblies using Crossgen2.")
Expand Down Expand Up @@ -194,6 +195,10 @@ handle_arguments_local() {
__TestBuildMode=crossgen2
;;

nativeaot|-nativeaot)
__TestBuildMode=nativeaot
;;

perfmap|-perfmap)
__CreatePerfmap=1
;;
Expand Down
10 changes: 10 additions & 0 deletions src/tests/run.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ if /i "%1" == "ilasmroundtrip" (set __IlasmRoundTrip=1&

if /i "%1" == "printlastresultsonly" (set __PrintLastResultsOnly=1&shift&goto Arg_Loop)
if /i "%1" == "runcrossgen2tests" (set RunCrossGen2=true&shift&goto Arg_Loop)
if /i "%1" == "runnativeaottests" (set RunNativeAot=true&shift&goto Arg_Loop)
if /i "%1" == "nativeaotmultimodule" (set NativeAotMultimodule=true&shift&goto Arg_Loop)
REM This test feature is currently intentionally undocumented
if /i "%1" == "runlargeversionbubblecrossgen2tests" (set RunCrossGen2=true&set CrossgenLargeVersionBubble=true&shift&goto Arg_Loop)
if /i "%1" == "link" (set DoLink=true&set ILLINK=%2&shift&shift&goto Arg_Loop)
Expand Down Expand Up @@ -139,6 +141,14 @@ if defined CrossgenLargeVersionBubble (
set __RuntestPyArgs=%__RuntestPyArgs% --large_version_bubble
)

if defined RunNativeAot (
set __RuntestPyArgs=%__RuntestPyArgs% --run_nativeaot_tests
)

if defined NativeAotMultimodule (
set __RuntestPyArgs=%__RuntestPyArgs% --nativeaot_multimodule
)

if defined __PrintLastResultsOnly (
set __RuntestPyArgs=%__RuntestPyArgs% --analyze_results_only
)
Expand Down
Loading