Skip to content

Run CoreCLR runtime tests using Mono with LLVM AOT. #38547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 14, 2020
Merged

Run CoreCLR runtime tests using Mono with LLVM AOT. #38547

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
2f390b4
Draft: run CoreCLR runtime tests using Mono with LLVM AOT.
imhameed Jun 29, 2020
af7d6cc
Temporarily avoid using the runtimeVariant string when defining coreC…
imhameed Jun 30, 2020
d919fcf
Compile the runtime tests using LLVM AOT in a simple configuration wh…
imhameed Jun 30, 2020
1ba1131
Use --tag=CXX when linking libmonosgen-2.0.so via libtool when LLVM i…
imhameed Jul 7, 2020
8a7cc0c
Mark function DISubprograms as local definitions.
imhameed Jul 7, 2020
a394e71
Disable IncompatibleOverride on Mono.
imhameed Jul 8, 2020
afee138
Add tests that fail after LLVM AOT compilation to issues.targets.
imhameed Jul 8, 2020
fb01596
Only rebuild Mono with LLVM AOT support when mono or the runtime test…
imhameed Jul 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions eng/pipelines/common/templates/runtimes/run-test-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,10 @@ jobs:
/p:TargetArchitecture=$(archType)
displayName: "Patch dotnet with mono"

- ${{ if and(eq(parameters.runtimeFlavor, 'mono'), eq(parameters.runtimeVariant, 'llvmaot')) }}:
- script: $(coreClrRepoRootDir)build-test$(scriptExt) mono_aot $(buildConfig)
displayName: "LLVM AOT compile CoreCLR tests"

# Send tests to Helix
- template: /eng/pipelines/common/templates/runtimes/send-to-helix-step.yml
parameters:
Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/mono/templates/xplat-pipeline-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:

variables:
- name: coreClrProductArtifactName
value: 'CoreCLRProduct_${{ parameters.runtimeVariant }}_$(osGroup)$(osSubgroup)_$(archType)_${{ parameters.liveRuntimeBuildConfig }}'
value: 'CoreCLRProduct__$(osGroup)$(osSubgroup)_$(archType)_${{ parameters.liveRuntimeBuildConfig }}'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this now shared or why are we changing this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done to allow LLVM AOT Mono-specific jobs to use the one-and-only variant of CoreCLR that we build right now.

Support for CoreCLR runtimeVariants was added in #35841. But runtimeVariant isn't currently used at all when building CoreCLR. The choices I see right now are:

  • remove runtimeVariant support entirely from eng/pipelines/coreclr/*.yml;
  • build an llvmaot variant of CoreCLR, for later consumption by LLVM AOT Mono-specific jobs, that is identical to the non-variant CoreCLR; or
  • explicitly depend on a shared non-variant CoreCLR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "explicitly depend on a shared non-variant CoreCLR." is the one that makes the most sense. I was mostly courious why was this change needed just to understand how the system worked.

It seems like we do have an extra _ in between CoreCLRProduct_ and osGroup though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "explicitly depend on a shared non-variant CoreCLR." is the one that makes the most sense. I was mostly courious why was this change needed just to understand how the system worked.

Ah, got it.

It seems like we do have an extra _ in between CoreCLRProduct_ and osGroup though.

Yeah. That's a consequence of how the product artifact name is concatenated: https://github.com/dotnet/runtime/pull/35841/files#diff-9d649abd6ea7b7db8a63e5dd8edfd14bR81

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to block this PR because of that since it seems like it is something we already heavily depend on and the change might be bigger that it seems.


- name: coreClrProductRootFolderPath
value: '$(Build.SourcesDirectory)/artifacts/bin/coreclr/$(osGroup).$(archType).$(liveRuntimeBuildConfigUpper)'
Expand Down
48 changes: 45 additions & 3 deletions eng/pipelines/runtime.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ jobs:
#
# Build CoreCLR checked using GCC toolchain
# Only when CoreCLR is changed
#
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/build-job.yml
Expand Down Expand Up @@ -250,7 +250,7 @@ jobs:
or(
eq(dependencies.checkout.outputs['SetPathVars_coreclr.containsChange'], true),
eq(variables['isFullMatrix'], true))

# Build the whole product using Mono runtime
# Only when libraries, mono or installer are changed
#
Expand Down Expand Up @@ -479,6 +479,25 @@ jobs:
eq(dependencies.checkout.outputs['SetPathVars_mono.containsChange'], true),
eq(variables['isFullMatrix'], true))

#
# Build Mono release with LLVM AOT
# Only when mono, or the runtime tests changed
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/mono/templates/build-job.yml
runtimeFlavor: mono
buildConfig: release
platforms:
- Linux_x64
jobParameters:
runtimeVariant: llvmaot
condition: >-
or(
eq(dependencies.checkout.outputs['SetPathVars_runtimetests.containsChange'], true),
eq(dependencies.checkout.outputs['SetPathVars_mono.containsChange'], true),
eq(variables['isFullMatrix'], true))

#
# Build libraries using live CoreLib
# These set of libraries are built always no matter what changed
Expand Down Expand Up @@ -613,7 +632,7 @@ jobs:
isOfficialBuild: false
liveRuntimeBuildConfig: release
testScope: innerloop
condition: >-
condition: >-
or(
eq(dependencies.checkout.outputs['SetPathVars_libraries.containsChange'], true),
eq(dependencies.checkout.outputs['SetPathVars_coreclr.containsChange'], true),
Expand Down Expand Up @@ -745,6 +764,29 @@ jobs:
eq(dependencies.checkout.outputs['SetPathVars_mono.containsChange'], true),
eq(dependencies.checkout.outputs['SetPathVars_runtimetests.containsChange'], true),
eq(variables['isFullMatrix'], true))
#
# Mono CoreCLR runtime Test executions using live libraries and LLVM AOT
# Only when Mono is changed
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/templates/runtimes/run-test-job.yml
buildConfig: release
runtimeFlavor: mono
platforms:
- Linux_x64
helixQueueGroup: pr
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: innerloop
liveLibrariesBuildConfig: ${{ variables.debugOnPrReleaseOnRolling }}
liveRuntimeBuildConfig: release
runtimeVariant: llvmaot
condition: >-
or(
eq(dependencies.checkout.outputs['SetPathVars_mono.containsChange'], true),
eq(dependencies.checkout.outputs['SetPathVars_runtimetests.containsChange'], true),
eq(variables['isFullMatrix'], true))

#
# Libraries Release Test Execution against a release mono runtime.
Expand Down
24 changes: 23 additions & 1 deletion src/coreclr/build-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,18 @@ build_test_wrappers()
fi
}

build_mono_aot()
{
__RuntimeFlavor="mono"
__MonoBinDir="$__RootBinDir/bin/mono/$__TargetOS.$__BuildArch.$__BuildType"
__Exclude="${__ProjectDir}/tests/issues.targets"
__TestBinDir="$__TestWorkingDir"
CORE_ROOT="$__TestBinDir"/Tests/Core_Root
export __Exclude
export CORE_ROOT
build_MSBuild_projects "Tests_MonoAot" "$__ProjectDir/tests/src/runtest.proj" "Mono AOT compile tests" "/t:MonoAotCompileTests" "/p:RuntimeFlavor=$__RuntimeFlavor" "/p:MonoLlvmPath=$__MonoBinDir"
}

generate_layout()
{
echo "${__MsgPrefix}Creating test overlay..."
Expand Down Expand Up @@ -608,6 +620,12 @@ handle_arguments_local() {
excludemonofailures|-excludemonofailures)
__Mono=1
;;

mono_aot|-mono_aot)
__Mono=1
__MonoAot=1
;;

*)
__UnprocessedBuildArgs+=("$1")
;;
Expand Down Expand Up @@ -661,6 +679,8 @@ __UseNinja=0
__VerboseBuild=0
__CMakeArgs=""
__priority1=
__Mono=0
__MonoAot=0
CORE_ROOT=

source "$__ProjectRoot"/_build-commons.sh
Expand Down Expand Up @@ -703,10 +723,12 @@ if [[ -z "$HOME" ]]; then
echo "HOME not defined; setting it to $HOME"
fi

if [[ (-z "$__GenerateLayoutOnly") && (-z "$__BuildTestWrappersOnly") ]]; then
if [[ (-z "$__GenerateLayoutOnly") && (-z "$__BuildTestWrappersOnly") && ("$__MonoAot" -eq 0) ]]; then
build_Tests
elif [[ ! -z "$__BuildTestWrappersOnly" ]]; then
build_test_wrappers
elif [[ "$__MonoAot" -eq 1 ]]; then
build_mono_aot
else
generate_layout
fi
Expand Down
89 changes: 89 additions & 0 deletions src/coreclr/tests/issues.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -984,6 +984,95 @@

<!-- Known failures for mono runtime on *all* architectures/operating systems -->
<ItemGroup Condition="'$(RuntimeFlavor)' == 'mono'" >
<ExcludeList Include="$(XunitTestBinBase)/Interop/StringMarshalling/AnsiBSTR/AnsiBStrTest/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/Interop/StringMarshalling/BSTR/BSTRTest/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/Regression/GitHub_17073/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>

<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/Sse3/LoadDquVector128_r/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/Sse3/LoadDquVector128_ro/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>

<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/General/VectorArray_r/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/General/VectorArray_ro/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/General/VectorRet_r/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/General/VectorRet_ro/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/General/VectorUnused_r/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/X86/General/VectorUnused_ro/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>

<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/General/Vector128/Vector128_ro/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/HardwareIntrinsics/General/Vector128/Vector128_r/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/Loader/classloader/MethodImpl/CovariantReturns/Structs/IncompatibleOverride/**">
<Issue>Crashes during LLVM AOT compilation.</Issue>
</ExcludeList>

<ExcludeList Include="$(XunitTestBinBase)/JIT/opt/InstructionCombining/DivToMul/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/SIMD/VectorConvert_ro_Target_64Bit/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/SIMD/VectorConvert_r_Target_64Bit/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/Directed/StructABI/StructABI/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/Regression/JitBlue/GitHub_15949/GitHub_15949/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/Regression/JitBlue/GitHub_15291/GitHub_15291/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/Regression/CLR-x86-JIT/V1-M09.5-PDC/b16928/b16928/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/jit64/opt/osr/osr015/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/JIT/jit64/opt/rngchk/ArrayBound_o/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/Loader/classloader/TypeInitialization/CctorsWithSideEffects/CctorThrowInlinedStatic/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/Loader/classloader/MethodImpl/CovariantReturns/UnitTest/CompatibleWithTest/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/Loader/classloader/TypeInitialization/CctorsWithSideEffects/CctorThrowMethodAccess/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/Loader/AssemblyLoadContext30Extensions/AssemblyLoadContext30Extensions/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>
<ExcludeList Include="$(XunitTestBinBase)/Loader/classloader/MethodImpl/CovariantReturns/UnitTest/CompatibleWithTest/**">
<Issue>Doesn't pass after LLVM AOT compilation.</Issue>
</ExcludeList>

<ExcludeList Include="$(XunitTestBinBase)/GC/Features/Finalizer/finalizeother/finalizeexcep/**">
<Issue>PlatformDetection.IsPreciseGcSupported false on mono</Issue>
</ExcludeList>
Expand Down
36 changes: 36 additions & 0 deletions src/coreclr/tests/src/runtest.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,37 @@ namespace $([System.String]::Copy($(Category)).Replace(".","_").Replace("\","").
</ItemGroup>
</Target>

<Target Name="MonoAotCompileTests" DependsOnTargets="GetListOfTestCmds;FindCmdDirectories">
<ItemGroup>
<AllTestScripts Include="%(TestDirectories.Identity)\**\*.sh" />
<TestExclusions Include="@(ExcludeList->Metadata('FullPath'))" Condition="$(HaveExcludes)" />
<TestScripts Include="@(AllTestScripts)" Exclude="@(TestExclusions)" />
<TestDllPaths Include="$([System.IO.Path]::ChangeExtension('%(TestScripts.Identity)', 'dll'))" />
<TestDlls Include="%(TestDllPaths.Identity)" Condition="Exists(%(TestDllPaths.Identity))" />
</ItemGroup>
<PropertyGroup>
<CorerunExecutable Condition="'$(RunningOnUnix)' == 'true'">$(CORE_ROOT)\corerun</CorerunExecutable>
<CorerunExecutable Condition="'$(RunningOnUnix)' != 'true'">$(CORE_ROOT)\corerun.exe</CorerunExecutable>
</PropertyGroup>
<ItemGroup>
<AotEnvVar Include="MONO_ENV_OPTIONS=--aot=llvm,llvm-path='$(MonoLlvmPath)'" />
</ItemGroup>
<PropertyGroup>
<AotEnvVars>@(AotEnvVar)</AotEnvVars>
</PropertyGroup>
<ItemGroup>
<AotProject Include="../../../mono/msbuild/aot-compile.proj">
<Properties>_CorerunExecutable=$(CorerunExecutable);_TestDll=%(TestDlls.Identity);AotEnvVars=@(AotEnvVar)</Properties>
</AotProject>
</ItemGroup>
<MSBuild
Projects="@(AotProject)"
Targets="AotCompile"
Condition="@(TestDlls->Count()) &gt; 0"
BuildInParallel="true"
/>
</Target>

<Target Name="CreateAllWrappers" DependsOnTargets="GetListOfTestCmds;FindCmdDirectories">
<MSBuild Projects="$(MSBuildProjectFile)" Targets="CreateXunitWrapper;BuildXunitWrapper" Properties="_CMDDIR=%(TestDirectories.Identity);TestSrcDir=$(XunitTestBinBase)" />
</Target>
Expand All @@ -374,6 +405,11 @@ namespace $([System.String]::Copy($(Category)).Replace(".","_").Replace("\","").
<MSBuild Projects="$(MSBuildThisFileDirectory)Common\test_dependencies\test_dependencies.csproj"
Condition=" '$(BuildWrappers)'=='true' " />

<MSBuild Projects="$(MSBuildProjectFile)"
Targets="MonoAotCompileTests"
Properties=""
Condition=" '$(MonoAotCompile)'=='true' " />

<!-- Default for building -->
<MSBuild Projects="$(MSBuildProjectFile)"
Targets="CreateAllWrappers"
Expand Down
2 changes: 2 additions & 0 deletions src/mono/configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6767,8 +6767,10 @@ AC_SUBST(MONO_NATIVE_PLATFORM_TYPE_UNIFIED)
#
if test "x$enable_llvm_runtime" = "xyes"; then
AC_SUBST(MONO_CXXLD, [$CXX])
AC_SUBST(MONO_LIBTOOL_TAG, '--tag=CXX')
else
AC_SUBST(MONO_CXXLD, [$CC])
AC_SUBST(MONO_LIBTOOL_TAG, '')
fi

### Set -Werror options
Expand Down
1 change: 1 addition & 0 deletions src/mono/mono/mini/Makefile.am.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -766,6 +766,7 @@ libmonosgen_2_0_la_CFLAGS = $(mono_sgen_CFLAGS) @CXX_ADD_CFLAGS@

libmonosgen_2_0_la_LIBADD = libmini.la $(interp_libs_with_mini) $(dbg_libs_with_mini) $(sgen_libs) $(LIBMONO_DTRACE_OBJECT) $(LLVMMONOF)
libmonosgen_2_0_la_LDFLAGS = $(libmonoldflags) $(monobin_platform_ldflags) $(CCLDFLAGS)
libmonosgen_2_0_la_LIBTOOLFLAGS = $(MONO_LIBTOOL_TAG)

noinst_LIBRARIES += libmaintest.a

Expand Down
4 changes: 3 additions & 1 deletion src/mono/mono/mini/mini-llvm-cpp.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -520,7 +520,9 @@ mono_llvm_di_create_function (void *di_builder, void *cu, LLVMValueRef func, con
di_file = builder->createFile (file, dir);
type = builder->createSubroutineType (builder->getOrCreateTypeArray (ArrayRef<Metadata*> ()));
#if LLVM_API_VERSION >= 900
di_func = builder->createFunction (di_file, name, mangled_name, di_file, line, type, 0);
di_func = builder->createFunction (
di_file, name, mangled_name, di_file, line, type, 0,
DINode::FlagZero, DISubprogram::SPFlagDefinition | DISubprogram::SPFlagLocalToUnit);
#else
di_func = builder->createFunction (di_file, name, mangled_name, di_file, line, type, true, true, 0);
#endif
Expand Down
6 changes: 6 additions & 0 deletions src/mono/msbuild/aot-compile.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="AotCompile">
<Exec Command="$(_CorerunExecutable) $(_TestDll)" EnvironmentVariables="$(AotEnvVars)" />
</Target>
</Project>