Build-test.sh handles native test assets #19430

AaronRobinsonMSFT · 2018-08-11T00:05:53Z

See #19416

…E flags for building tests

Remove compileoptions.cmake and push contents into configurecompiler.cmake which contains compiler configuration from repo root CMakeLists.txt

AaronRobinsonMSFT · 2018-08-11T00:06:24Z

@jashook or @BruceForstall Can one of you kick off the ARM series of tests?

…p tests

AaronRobinsonMSFT · 2018-08-11T01:49:59Z

@dotnet-bot test Windows_NT x86 Release Innerloop Build and Test

4creators · 2018-08-14T15:08:02Z

build-test.sh

@@ -152,7 +152,7 @@ generate_layout()
    # ===
    # =========================================================================================

-    build_Tests_internal "Restore_Packages" "${__ProjectDir}/tests/build.proj" "Restore product binaries (build tests)" "-BatchRestorePackages"
+    build_MSBuild_projects "Restore_Packages" "${__ProjectDir}/tests/build.proj" "Restore product binaries (build tests)" "-BatchRestorePackages"


After checking you PR I have noticed that default settings in config.json for invoking msbuild on unixes do not set maxcpucount parameter what makes all calls non parallel and slows down every msbuild based operation.

Another problem is related to node reuse. AFAIR node reuse was off by default on Linux, however, this could have changed now. In such case we would need -nodereuse=false msbuild parameter. @rainersigwald may know more about current state of node reuse support on Linux.

dotnet build and dotnet msbuild unconditionally insert -maxcpucount into MSBuild's command line.

Node reuse is on by default on all platforms since .NET Core SDK 2.1.300/MSBuild 15.7. Since there are many individual invocations of MSBuild here, I would expect it to provide a performance improvement and wouldn't recommend leaving it off entirely unless there's a pressing need. It might be a good idea to do a dotnet build-server shutdown at the end of the script (even on error paths), depending on the use case.

I didn't change any MSBuild logic in this review. That is definitely something we should investigate going forward though.

AaronRobinsonMSFT · 2018-08-14T20:56:42Z

@dotnet-bot test Windows_NT arm64 Cross Debug Innerloop Build
@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test
@dotnet-bot test Ubuntu arm Cross Checked Innerloop Build and Test

AaronRobinsonMSFT · 2018-08-14T20:57:18Z

@dotnet-bot tests Ubuntu arm Cross Checked no_tiered_compilation_innerloop Build and Test

AaronRobinsonMSFT · 2018-08-14T20:57:50Z

@dotnet-bot test Ubuntu arm Cross Checked no_tiered_compilation_innerloop Build and Test

jashook · 2018-08-14T20:59:27Z

@dotnet-bot test Windows_NT arm64 Cross Checked Innerloop Build and Test
@dotnet-bot test Windows_NT arm Cross Checked Innerloop Build and Test

AaronRobinsonMSFT · 2018-08-15T00:48:41Z

@jashook Any thoughts on why the ARMs are taking so long?
@RussKeldorph Based on the discovered test bug I am much more confident in this PR. Any concerns?

adityamandaleeka · 2018-08-15T02:26:42Z

@AaronRobinsonMSFT Looks like the ARM pool has dwindled down to just a couple of clients again so the job queue for them has ballooned, causing long delays. @jashook can get the dropped clients back online.

jashook · 2018-08-15T02:44:54Z

I have brought a few of them back online; however, the rest are unreachable and will need an os refresh.

RussKeldorph · 2018-08-15T15:03:18Z

@AaronRobinsonMSFT No objections. Can you list the configurations you tried in your private testing? For example, do the ARM/ARM64 cross builds build the expected components correctly?

AaronRobinsonMSFT · 2018-08-15T15:54:50Z

@RussKeldorph I verified Debug/Checked on Windows and MacOS. I didn't do any validation for ARM/ARM64, but I can. I assume I just pass the ARM flag to the respective build scripts?

jashook

Looks good to me after a first pass, want to look at it one more time later today.

RussKeldorph · 2018-08-15T16:04:00Z

Instructions for cross building on Unix are here. Essentially you have to build a rootfs and then build with the cross and architecture switches with ROOTFS_DIR set. Assuming this doesn't "just work," I'm ok with commit this PR as is (with appropirate caveat added to the description) and an additional issue opened to handle cross builds.

janvorli · 2018-08-15T16:07:54Z

I assume I just pass the ARM flag to the respective build scripts?

No, it is not that simple. You need to build rootfs for ARM and ARM64 first and then add cross arm or cross arm64 to the build.sh options.
To build the rootfs, run sudo cross/build-rootfs.sh arm xenial and sudo cross/build-rootfs.sh arm64 xenial. The xenial is optional, I think I used to have some problems with going with the default (trusty) for ARM / ARM64, but I am not sure.

jashook · 2018-08-15T16:10:51Z

The easiest way to do these builds currently is by using docker. An example build for arm and arm64 would be:

Armhf

docker run -i --rm -v <path/to/coreclr/repo/>:/coreclr -w /coreclr -e ROOTFS_DIR=/crossrootfs/arm microsoft/dotnet-buildtools-prereqs:ubuntu-14.04-cross-e435274-20180426002420 /coreclr/build.sh arm cross checked

docker run -i --rm -v <path/to/coreclr/repo/>:/coreclr -w /coreclr -e ROOTFS_DIR=/crossrootfs/arm microsoft/dotnet-buildtools-prereqs:ubuntu-14.04-cross-e435274-20180426002420 /coreclr/build-test.sh arm checked -priority=1

Arm64

Build product

docker run -i --rm -v <path/to/coreclr/repo/>:/coreclr -w /coreclr -e ROOTFS_DIR=/crossrootfs/arm64 microsoft/dotnet-buildtools-prereqs:ubuntu-16.04-cross-arm64-a3ae44b-20180315221921 /coreclr/build.sh arm64 cross checked

Build tests

docker run -i --rm -v <path/to/coreclr/repo/>:/coreclr -w /coreclr -e ROOTFS_DIR=/crossrootfs/arm64 microsoft/dotnet-buildtools-prereqs:ubuntu-16.04-cross-arm64-a3ae44b-20180315221921 /coreclr/build-test.sh arm64 checked -priority=1

jashook · 2018-08-15T16:11:45Z

It avoids having to go through our rootfs process, which is time consuming and error prone.

jashook

lgtm. I am good without running arm/arm64. As mentioned I think the following are worth validating before checking in:

pri1 run of all jobs
manual run of build-test.sh -priority=1 && runtest.sh ...

Thanks for the work!

AaronRobinsonMSFT · 2018-08-16T03:11:57Z

Was able to fix the script to do the count check.

Ran and passed with priority 1 on MacOS:

=======================
     Test Results
=======================
# CoreCLR Bin Dir  : 
# Tests Discovered : 12400
# Passed           : 11610
# Failed           : 0
# Skipped          : 790
=======================
41 minutes and 38 seconds taken to run CoreCLR tests.

jashook · 2018-08-16T06:34:30Z

@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test
@dotnet-bot test Ubuntu arm Cross Checked Innerloop Build and Test
@dotnet-bot test Windows_NT arm64 Cross Checked normal Build and Test
@dotnet-bot test Windows_NT arm Cross Checked normal Build and Test
@dotnet-bot test Windows_NT x64 Build and Test
@dotnet-bot test Windows_NT x86 Build and Test
@dotnet-bot test Ubuntu x64 Build and Test
@dotnet-bot test OSX10.12 x64 Build and Test

AaronRobinsonMSFT · 2018-08-16T16:31:00Z

@dotnet-bot test Ubuntu arm Cross Checked Innerloop Build and Test

BruceForstall · 2018-08-18T00:02:15Z

It appears this broke some test builds. I opened https://github.com/dotnet/coreclr/issues/19540 to track.

BruceForstall · 2018-08-18T00:29:11Z

btw, it looks like the trigger phrases used above to try to trigger Windows x86/x64 pri-1 runs didn't work, and the x86 trigger phrase was incorrect. This should work:

@dotnet-bot test Windows_NT x86 Checked Build and Test
@dotnet-bot test Windows_NT x64 Build and Test

AaronRobinsonMSFT added 7 commits August 8, 2018 11:02

Respect Windows script argument to skip package building.

973834f

Bring build.sh logic closer to build.cmd with respect to passing CMAK…

0b29717

…E flags for building tests

Rename build_Tests_internal to build_MSBuild_projects

2cfd120

Update comments regarding test logic

7ecdb0f

Make cmake gen script find override file without using script arguments

a186471

build-test.sh supports building native projects

8dc11d3

build-test.sh can now build native test projects

53e528f

Remove compileoptions.cmake and push contents into configurecompiler.cmake which contains compiler configuration from repo root CMakeLists.txt

AaronRobinsonMSFT added area-Build test enhancement Improvements of test source code labels Aug 11, 2018

AaronRobinsonMSFT added this to the 3.0 milestone Aug 11, 2018

jashook self-requested a review August 11, 2018 00:22

Fix Windows test build - always include ole32 and advapi32 for Intero…

736fce9

…p tests

AaronRobinsonMSFT added 3 commits August 10, 2018 20:26

Force file to be compiled as C

839847a

Compile all files as C for StructABI project

c3d400d

Use interop macro for CoTaskMem* functions

e69a35e

4creators reviewed Aug 14, 2018

View reviewed changes

Remove bug in test

8973f99

RussKeldorph approved these changes Aug 15, 2018

View reviewed changes

jashook reviewed Aug 15, 2018

View reviewed changes

jashook approved these changes Aug 15, 2018

View reviewed changes

Fix CheckTestBuild target run in build-test.sh

9df8794

AaronRobinsonMSFT merged commit f0f42d4 into dotnet:master Aug 16, 2018

AaronRobinsonMSFT deleted the build_test_sh_handles_native branch August 16, 2018 18:22

AaronRobinsonMSFT mentioned this pull request Aug 18, 2018

Enable mixed mode assembly loading #19542

Merged

sandreenko mentioned this pull request Aug 29, 2018

fix merge conflict in build-test.sh #19744

Merged

jashook added area-Infrastructure-coreclr and removed area-Build labels Nov 11, 2019

BruceForstall mentioned this pull request Jan 31, 2020

Windows x86 tests failing due to missing native dll entrypoints dotnet/runtime#10920

Closed

Build-test.sh handles native test assets #19430

Build-test.sh handles native test assets #19430

Uh oh!

Conversation

AaronRobinsonMSFT commented Aug 11, 2018

Uh oh!

AaronRobinsonMSFT commented Aug 11, 2018

Uh oh!

AaronRobinsonMSFT commented Aug 11, 2018

Uh oh!

4creators Aug 14, 2018

Choose a reason for hiding this comment

Uh oh!

rainersigwald Aug 14, 2018

Choose a reason for hiding this comment

Uh oh!

AaronRobinsonMSFT Aug 14, 2018

Choose a reason for hiding this comment

Uh oh!

AaronRobinsonMSFT commented Aug 14, 2018

Uh oh!

AaronRobinsonMSFT commented Aug 14, 2018

Uh oh!

AaronRobinsonMSFT commented Aug 14, 2018

Uh oh!

jashook commented Aug 14, 2018

Uh oh!

AaronRobinsonMSFT commented Aug 15, 2018

Uh oh!

adityamandaleeka commented Aug 15, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jashook commented Aug 15, 2018

Uh oh!

RussKeldorph commented Aug 15, 2018

Uh oh!

AaronRobinsonMSFT commented Aug 15, 2018

Uh oh!

jashook left a comment

Choose a reason for hiding this comment

Uh oh!

RussKeldorph commented Aug 15, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

janvorli commented Aug 15, 2018

Uh oh!

jashook commented Aug 15, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Armhf

Arm64

Build product

Build tests

Uh oh!

jashook commented Aug 15, 2018

Uh oh!

jashook left a comment

Choose a reason for hiding this comment

Uh oh!

AaronRobinsonMSFT commented Aug 16, 2018

Uh oh!

jashook commented Aug 16, 2018

Uh oh!

AaronRobinsonMSFT commented Aug 16, 2018

Uh oh!

BruceForstall commented Aug 18, 2018

Uh oh!

BruceForstall commented Aug 18, 2018

Uh oh!

Uh oh!

adityamandaleeka commented Aug 15, 2018 •

edited

Loading

RussKeldorph commented Aug 15, 2018 •

edited

Loading

jashook commented Aug 15, 2018 •

edited

Loading