Make build scripts' -help information more helpful #37377

ivdiazsa · 2020-06-04T01:05:16Z

This PR slightly changes the layout and information shown when asking for help to runtime's build scripts. The following changes were made:

Flags were sorted alphabetically, as most command-line tools do.
Short flags are now shown right next to their long counterparts.
Added the default values (if any) when a flag is omitted.
Added a few simple examples at the end.
Added a special handler when requesting -subset help. This is to optimize and avoid passing unnecessary data as this command does not want to trigger any build. Also, it adds a specific flag to prevent MSBuild from showing build metrics in this case.
Reformatted the lines a bit for better readability on smaller terminals/larger fonts.

These changes were applied to both, build.cmd and build.sh.

In addition to these changes, the "error" message shown when requesting -subset help was reworded to reassure the user it is not an error as such. The reason this mechanism is kept, is that since subset information is printed directly by MSBuild, there needs to be a way to stop it right after showing the subsets' descriptions. If we remove this, then restore finishes doing its thing and the descriptions are printed thrice. We don't want to spam the user either. If -restore is removed from the MSBuild command, it fails unless the build script has invoked restore in the past. This would cause an unknown error to appear when working with a clean repository, which is fairly often.

message softer and without metrics.

ghost · 2020-06-04T01:05:21Z

Tagging subscribers to this area: @ViktorHofer
Notify danmosemsft if you want to be subscribed.

danmoseley · 2020-06-04T01:12:27Z

Can you please paste in the new output so we can see what it looks like?

safern · 2020-06-04T01:16:20Z

eng/build.sh

      else
+        passedSubset="$(echo "$2" | awk '{print tolower($0)}')"


We also have detection on the first argument to parse it as a subset: https://github.com/dotnet/runtime/pull/37377/files#diff-75d32cbca8eddea951ce0484aa996d8fR130

So we also need to consider the case where people passes down:

.\build.sh help

Just tested it. It works and shows the new subset message, but it shows the metrics. Let me add a handler for that case as well.

ivdiazsa · 2020-06-04T01:19:01Z

Can you please paste in the new output so we can see what it looks like?

Sure, here are the outputs:

New Subset Help Message

\runtime\eng\SubsetValidation.targets(42,5): error : This is not an error. These are the available subsets. You can choose one or none at all for a full build. [C:\Git\runtime\Build.proj]

Build.cmd

Common settings:
  -arch (-a)                     Build platform: x86, x64, arm, arm64, or wasm.
                                 Pass a comma-separated list to build for multiple architectures.
                                 [Default: Your machine's architecture.]
  -binaryLog (-bl)               Output binary log.
  -configuration (-c)            Build configuration: Debug, Release or [CoreCLR]Checked.
                                 Pass a comma-separated list to build for multiple configurations.
                                 [Default: Debug]
  -help (-h)                     Print help and exit.
  -librariesConfiguration (-lc)  Libraries build configuration: Debug or Release.
                                 [Default: Debug]
  -os                            Build operating system: Windows_NT, Linux, OSX, or Browser.
                                 [Default: Your machine's OS.]
  -runtimeConfiguration (-rc)    Runtime build configuration: Debug, Release or [CoreCLR]Checked.
                                 [Default: Debug]
  -subset (-s)                   Build a subset, print available subsets with -subset help.
                                 '-subset' can be omitted if the subset is given as the first argument.
                                 [Default: Builds the entire repo.]
  -verbosity (-v)                MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic].
                                 [Default: Minimal]
  -vs                            Open the solution with VS using the locally acquired SDK. Path or solution name.
                                 (Example: -vs Microsoft.CSharp)

Actions (defaults to -restore -build):
  -build (-b)             Build all source projects.
  -clean                  Clean the solution.
  -pack                   Package build outputs into NuGet packages.
  -publish                Publish artifacts (e.g. symbols).
  -rebuild                Rebuild all source projects.
  -restore                Restore dependencies.
  -sign                   Sign build outputs.
  -test (-t)              Build and run tests.

Libraries settings:
  -allconfigurations      Build packages for all build configurations.
  -coverage               Collect code coverage when testing.
  -framework (-f)         Build framework: net5.0 or net472.
  -testnobuild            Skip building tests when invoking -test.
  -testscope              Scope tests, allowed values: innerloop, outerloop, all.

Command-line arguments not listed above are passed through to MSBuild.
The above arguments can be shortened as much as to be unambiguous.
(Example: -con for configuration, -t for test, etc.).

Here are some quick examples:

Build CoreCLR on Windows for x64 on release configuration:
.\build.cmd clr -os Windows_NT -arch x64 -configuration Release

Cross-compile CoreCLR runtime on Windows for ARM64.
.\build.cmd -subset clr.runtime -os Windows_NT -arch arm64 -configuration Release

Build Debug libraries with a Release runtime.
.\build.cmd -subset clr+libs -os Windows_NT -arch x64 -runtimeConfiguration Release

For more information, check out https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md

Build.sh

Common settings:
  --arch                          Build platform: x86, x64, arm, armel, arm64 or wasm.
                                  [Default: Your machine's architecture.]
  --binaryLog (-bl)               Output binary log.
  --cross                         Optional argument to signify cross compilation.
  --configuration (-c)            Build configuration: Debug, Release or [CoreCLR]Checked.
                                  [Default: Debug]
  --help (-h)                     Print help and exit.
  --librariesConfiguration (-lc)  Libraries build configuration: Debug or Release.
                                  [Default: Debug]
  --os                            Build operating system: Windows_NT, Linux, FreeBSD, OSX, tvOS, iOS, Android, Browser, NetBSD or SunOS.
                                  [Default: Your machine's OS.]
  --projects <value>              Project or solution file(s) to build.
  --runtimeConfiguration (-rc)    Runtime build configuration: Debug, Release or [CoreCLR]Checked.
                                  [Default: Debug]
  --subset (-s)                   Build a subset, print available subsets with -subset help.
                                 '--subset' can be omitted if the subset is given as the first argument.
                                  [Default: Builds the entire repo.]
  --verbosity (-v)                MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic].
                                  [Default: Minimal]

Actions (defaults to --restore --build):
  --build (-b)               Build all source projects.
  --clean                    Clean the solution.
  --pack                     Package build outputs into NuGet packages.
  --publish                  Publish artifacts (e.g. symbols).
  --rebuild                  Rebuild all source projects.
  --restore (-r)             Restore dependencies.
  --sign                     Sign build outputs.
  --test (-t)                Build and run tests.

Libraries settings:
  --allconfigurations        Build packages for all build configurations.
  --coverage                 Collect code coverage when testing.
  --framework (-f)           Build framework: net5.0 or net472.
  --testnobuild              Skip building tests when invoking -test.
  --testscope                Test scope, allowed values: innerloop, outerloop, all.

Native build settings:
  --clang                    Optional argument to build using clang in PATH (default).
  --clangx.y                 Optional argument to build using clang version x.y.
  --cmakeargs                User-settable additional arguments passed to CMake.
  --gcc                      Optional argument to build using gcc in PATH (default).
  --gccx.y                   Optional argument to build using gcc version x.y.

Command line arguments starting with '/p:' are passed through to MSBuild.
Arguments can also be passed in with a single hyphen.

Here are some quick examples:

Build CoreCLR on Linux for x64 on release configuration:
./build.sh clr --os linux --arch x64 --configuration release

Build Debug libraries with a Release runtime.
./build.sh --subset clr+libs --os linux --arch x64 --runtimeConfiguration release

Cross-compile CoreCLR runtime on Linux for ARM64.
./build.sh -subset clr.runtime --os linux --arch arm64 --configuration release --cross

However, for this last example, you need to already have ROOTFS_DIR set up.
Further information on this can be found here:
https://github.com/dotnet/runtime/blob/master/docs/workflow/building/coreclr/linux-instructions.md

For more general information, check out https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md

BruceForstall

This is a big improvement; thanks.

danmoseley · 2020-06-04T17:36:56Z

Thank you for adding examples. We have gotten a lot of feedback that it's too hard to figure out how to build. I often have to go back to the docs and I'm just copy/pasting examples from there. Whenever a command line tool has this many parameters and concepts, examples are super valuable. Are there any other common use cases it's worth adding examples for? I think it's fine to have more.

danmoseley · 2020-06-04T17:38:25Z

eng/build.sh

+  echo "Build CoreCLR on Linux for x64 on release configuration:"
+  echo "./build.sh clr --os linux --arch x64 --configuration release"
+  echo ""
+  echo "Build Debug libraries with a Release runtime."


Suggested change

echo "Build Debug libraries with a Release runtime."

echo "Build Debug libraries for Linux with a Release runtime."

danmoseley · 2020-06-04T17:38:54Z

eng/build.sh

+  echo ""
+  echo "Build Debug libraries with a Release runtime."
+  echo "./build.sh --subset clr+libs --os linux --arch x64 --runtimeConfiguration release"
+  echo ""


How about adding build -subset clr+libs -runtimeConfiguration Release (it's what we suggest for library work in docs\workflow\building\libraries\README.md)

And note that that one picks the same target platform you are building on

FWIW the shortest form (that most of us use) is build clr+libs -rc Release.

danmoseley · 2020-06-04T17:41:04Z

eng/build.sh

-  echo "  --clang                    Optional argument to build using clang in PATH (default)"
-  echo "  --clangx.y                 Optional argument to build using clang version x.y"
+  echo "  --clang                    Optional argument to build using clang in PATH (default)."
+  echo "  --clangx.y                 Optional argument to build using clang version x.y."


Can you add an example, eg., (e.g., --clang11.0) (is that right?)

Same for gcc

--clang11.0 would not be a good example, clang versions from 7 on use only major version numbers. So we could put in e.g. --clang9
Which actually reveals that the help is slightly incorrect.

@ivdiazsa I think this is still wrong? sounds like it should be --clangx ?

danmoseley · 2020-06-04T17:42:10Z

eng/build.sh

+  echo "Cross-compile CoreCLR runtime on Linux for ARM64."
+  echo "./build.sh -subset clr.runtime --os linux --arch arm64 --configuration release --cross"
+  echo ""
+  echo "However, for this last example, you need to already have ROOTFS_DIR set up."


more examples (5? ) would be great -- eg., a mono one.

ViktorHofer · 2020-06-04T17:44:58Z

eng/build.ps1

+  $argumentsForHelp = "-restore -build /p:subset=help /clp:nosummary"
+  Invoke-Expression "& `"$PSScriptRoot/common/build.ps1`" $argumentsForHelp"


nit: inline $argumentsForHelp into the Invoke-Expression statement.

ViktorHofer · 2020-06-04T17:46:28Z

eng/build.sh

+  argumentsForHelp="-restore -build /p:subset=help /clp:nosummary"
+  "$scriptroot/common/build.sh" $argumentsForHelp


Same comment about inlining.

danmoseley · 2020-06-04T17:54:48Z

eng/build.ps1

+  Write-Host "                                 Pass a comma-separated list to build for multiple architectures."
+  Write-Host "                                 [Default: Your machine's architecture.]"
+  Write-Host "  -binaryLog (-bl)               Output binary log."
+  Write-Host "  -configuration (-c)            Build configuration: Debug, Release or [CoreCLR]Checked."


Doesn't casing matter (at least on Unix)? @safern should it be consistently lower case or upper? We should note that in the help, and fix the help.

The casing doesn't matter as we are upper-casing passed-in configurations in the scripts.

Right. In a previous PR I made (35642) it no longer matters if you pass --os LINUX or --os linux. It gets handled internally and converted to the casing MSBuild expects.

consistently using lower case in examples seems better but I can't put my finger on why so it's probably just me :)

ViktorHofer · 2020-06-04T17:54:48Z

eng/build.ps1

+  Write-Host ".\build.cmd -subset clr.runtime -os Windows_NT -arch arm64 -configuration Release"
+  Write-Host ""
+  Write-Host "Build Debug libraries with a Release runtime."
+  Write-Host ".\build.cmd -subset clr+libs -os Windows_NT -arch x64 -runtimeConfiguration Release"


This is super verbose and I don't know anyone specifying the defaults for OS and Arch.

Suggested change

Write-Host ".\build.cmd -subset clr+libs -os Windows_NT -arch x64 -runtimeConfiguration Release"

Write-Host ".\build.cmd clr+libs -rc Release"

I agree, I never specify -os or -arch explicitly.

I mean, I do specify arch sometime when I want to do a cross build, but it is very rare.

I often specify arch when building on Windows x64 to build arm/arm64/x86.

One thing that is very confusing to me is: when do I need to specify -cross?

From what the docs say and my past experiences, --cross is a flag specified when you are doing a cross-build, like building for ARM64 on x64. Worth mentioning it's only necessary on Unix (i.e. build.sh)

danmoseley · 2020-06-04T17:55:47Z

eng/build.ps1

+  Write-Host "                                 [Default: Builds the entire repo.]"
+  Write-Host "  -verbosity (-v)                MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic]."
+  Write-Host "                                 [Default: Minimal]"
+  Write-Host "  -vs                            Open the solution with VS using the locally acquired SDK. Path or solution name."


Suggested change

Write-Host " -vs Open the solution with VS using the locally acquired SDK. Path or solution name."

Write-Host " -vs Open the solution with Visual Studio using the locally acquired SDK. Path or solution name."

Does this work with .csproj or only .sln @ViktorHofer ?

You can use it to open anything that works inside VS :)

OK then we should say solution or project file.

danmoseley · 2020-06-04T17:56:14Z

eng/build.ps1

-  Write-Host "  -allconfigurations      Build packages for all build configurations"
+  Write-Host "  -allconfigurations      Build packages for all build configurations."
+  Write-Host "  -coverage               Collect code coverage when testing."
+  Write-Host "  -framework (-f)         Build framework: net5.0 or net472."


Suggested change

Write-Host " -framework (-f) Build framework: net5.0 or net472."

Write-Host " -framework (-f) Target framework: net5.0 or net472. Defaults to net5.0."

danmoseley · 2020-06-04T17:56:47Z

eng/build.ps1

-  Write-Host "  -sign                   Sign build outputs"
-  Write-Host "  -publish                Publish artifacts (e.g. symbols)"
-  Write-Host "  -clean                  Clean the solution"
+  Write-Host "  -build (-b)             Build all source projects."


Suggested change

Write-Host " -build (-b) Build all source projects."

Write-Host " -build (-b) Build all source projects. (Assumes you already ran -restore)"

I made this mistake in the past

danmoseley · 2020-06-04T17:57:32Z

eng/build.ps1

+  Write-Host "  -build (-b)             Build all source projects."
+  Write-Host "  -clean                  Clean the solution."
+  Write-Host "  -pack                   Package build outputs into NuGet packages."
+  Write-Host "  -publish                Publish artifacts (e.g. symbols)."


Suggested change

Write-Host " -publish Publish artifacts (e.g. symbols)."

Write-Host " -publish Publish artifacts (e.g. symbols - assumes you already ran -build)."

danmoseley · 2020-06-04T17:58:14Z

eng/build.ps1

+  Write-Host "  -rebuild                Rebuild all source projects."
+  Write-Host "  -restore                Restore dependencies."
+  Write-Host "  -sign                   Sign build outputs."
+  Write-Host "  -test (-t)              Build and run tests."


Does this actually build tests, or just run?
Also, does it build the library under test, or just the test library?

good to note this.

That's an interesting observation and something worth including. I believe it just builds the test library, but I've not used it much so I'm not sure. Can you clarify this @janvorli?

the target test just runs the tests. In order to build you need to do -build and passing the right subset that includes tests.

The -test switch incrementally restores and builds the test projects. You can specify -test -testnobuild to only run the tests.

runtime/eng/build.ps1

Lines 108 to 113 in 6176924

# Check if an action is passed in

$actions = "b","build","r","restore","rebuild","sign","testnobuild","publish","clean"

$actionPassedIn = @(Compare-Object -ReferenceObject @($PSBoundParameters.Keys) -DifferenceObject $actions -ExcludeDifferent -IncludeEqual).Length -ne 0

if ($null -ne $properties -and $actionPassedIn -ne $true) {

$actionPassedIn = @(Compare-Object -ReferenceObject $properties -DifferenceObject $actions.ForEach({ "-" + $_ }) -ExcludeDifferent -IncludeEqual).Length -ne 0

}

Thanks for clarifying!

danmoseley · 2020-06-04T18:06:46Z

Thanks for doing this, one thing already clear from the community survey is that build is intimidating, and good help text has to help.

BruceForstall · 2020-06-04T18:25:54Z

One thing that it would be useful to describe in more detail, although in the documentation on GitHub and not in this help message, is the difference between -c, -lc, and -rc. It seems to me there are more subtleties here than are obvious: I always assumed that -c give the "default" and -lc/-rc overrides it, or that you could provide both -lc/-rc and not -c and it would be fine because nothing would use the value from -c, but I think that's not actually correct.

ivdiazsa · 2020-06-04T19:42:46Z

One thing that it would be useful to describe in more detail, although in the documentation on GitHub and not in this help message, is the difference between -c, -lc, and -rc. It seems to me there are more subtleties here than are obvious: I always assumed that -c give the "default" and -lc/-rc overrides it, or that you could provide both -lc/-rc and not -c and it would be fine because nothing would use the value from -c, but I think that's not actually correct.

I agree with this. It is not clear what are the implications of using each one, and under which scenarios somebody would use one or the other. I would gladly make this change but I'd need some guidance on how these flags work :)

ViktorHofer · 2020-06-04T19:47:40Z

but I'd need some guidance on how these flags work :)

As @BruceForstall described, passing in a configuration via -configuration/-c sets the base configuration which can either be Debug/Release or for CoreCLR Checked. If you specify -runtimeconfiguration/-rc, you control the configuration for the runtime projects (mono & coreclr) and it wins over the passed in configuration with the -configuration/-c switch. Same for -librariesconfiguration/-lc for libraries.

So specifying build clr+libs -rc Debug -c Release means that you will build the runtime projects (coreclr in this case) with Debug and everything else with Release (in this case only libs).

ivdiazsa · 2020-06-04T19:58:54Z

Thank you very much to everyone for the feedback on this! Now I have a clearer picture on the most common use cases for these build scripts, and how they work. I will recap the proposed changes in the following list, and please let me know if I missed something.

Changes to be done:

Make the examples less verbose (i.e. use short flags and omit os and arch in those where there is no cross-building involved.
Rephrase/clarify some wording:
- Change Build OS to Target OS.
- Specify Checked is only for CoreCLR.
- Specify assumptions for certain flags (e.g. -publish assumes -build has been run).
- Add missing defaults (e.g. -f defaults to net5.0).
- Other small changes like using "for Linux" instead of "on Linux" and changing "VS" to "Visual Studio".
Inline help command-line arguments.
Add more examples:
- Using --clang.
- Using --gcc.
- One on building Mono.
Handle special case ./build.sh help, which is assumed to be ./build.sh --subset help.

danmoseley · 2020-06-04T20:24:31Z

It would be great to see the updated output before we merge.

BruceForstall · 2020-06-04T22:00:54Z

@ViktorHofer

So specifying build clr+libs -rc Debug -c Release means that you will build the runtime projects (coreclr in this case) with Debug and everything else with Release (in this case only libs).

so do all of the following do the same thing?

build clr+libs+libs.tests -rc Debug -c Release
build clr+libs+libs.tests -c Debug -lc Release
build clr+libs+libs.tests -rc Debug -lc Release

safern · 2020-06-04T22:14:15Z

so do all of the following do the same thing?

Correct.

ivdiazsa · 2020-06-05T19:09:52Z

Here are the new updated versions of the help messages (updated with Bruce's latest observations):

Build.cmd

Common settings:
  -arch (-a)                     Target platform: x86, x64, arm, arm64, or wasm.
                                 Pass a comma-separated list to build for multiple architectures.
                                 [Default: Your machine's architecture.]
  -binaryLog (-bl)               Output binary log.
  -configuration (-c)            Build configuration: Debug, Release or Checked.
                                 Checked is exclusive to the CLR subset. It is the same as Debug, except code is
                                 compiled with optimizations enabled.
                                 Pass a comma-separated list to build for multiple configurations.
                                 [Default: Debug]
  -help (-h)                     Print help and exit.
  -librariesConfiguration (-lc)  Libraries build configuration: Debug or Release.
                                 [Default: Debug]
  -os                            Target operating system: Windows_NT, Linux, OSX, or Browser.
                                 [Default: Your machine's OS.]
  -runtimeConfiguration (-rc)    Runtime build configuration: Debug, Release or Checked.
                                 Checked is exclusive to the CLR runtime. It is the same as Debug, except code is
                                 compiled with optimizations enabled.
                                 [Default: Debug]
  -subset (-s)                   Build a subset, print available subsets with -subset help.
                                 '-subset' can be omitted if the subset is given as the first argument.
                                 [Default: Builds the entire repo.]
  -verbosity (-v)                MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic].
                                 [Default: Minimal]
  -vs                            Open the solution with Visual Studio using the locally acquired SDK.
                                 Path or any project or solution name is accepted.
                                 (Example: -vs Microsoft.CSharp)

Actions (defaults to -restore -build):
  -build (-b)             Build all source projects.
                          This assumes -restore has been run already.
  -clean                  Clean the solution.
  -pack                   Package build outputs into NuGet packages.
  -publish                Publish artifacts (e.g. symbols).
                          This assumes -build has been run already.
  -rebuild                Rebuild all source projects.
  -restore                Restore dependencies.
  -sign                   Sign build outputs.
  -test (-t)              Incrementally builds and runs tests.
                          Use in conjuction with -testnobuild to only run tests.

Libraries settings:
  -allconfigurations      Build packages for all build configurations.
  -coverage               Collect code coverage when testing.
  -framework (-f)         Build framework: net5.0 or net472.
                          [Default: net5.0]
  -testnobuild            Skip building tests when invoking -test.
  -testscope              Scope tests, allowed values: innerloop, outerloop, all.

Command-line arguments not listed above are passed through to MSBuild.
The above arguments can be shortened as much as to be unambiguous.
(Example: -con for configuration, -t for test, etc.).

Here are some quick examples. These assume you are on a Windows x64 machine:

* Build CoreCLR for Windows x64 on Release configuration:
.\build.cmd clr -c release

* Cross-compile CoreCLR runtime for Windows ARM64 on Release configuration.
.\build.cmd clr.runtime -arch arm64 -c release

* Build Debug libraries with a Release runtime for Windows x64.
.\build.cmd clr+libs -rc release

* Build Release libraries and their tests with a Checked runtime for Windows x64, and run the tests.
.\build.cmd clr+libs+libs.tests -rc checked -lc release -test

* Build Mono runtime for Windows x64 on Release configuration.
.\build.cmd mono -c release

It's important to mention that to build Mono for the first time,
you need to build the CLR and Libs subsets beforehand.
This is done automatically if a full build is performed at first.

For more information, check out https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md

Build.sh

Common settings:
  --arch                          Target platform: x86, x64, arm, armel, arm64 or wasm.
                                  [Default: Your machine's architecture.]
  --binaryLog (-bl)               Output binary log.
  --cross                         Optional argument to signify cross compilation.
  --configuration (-c)            Build configuration: Debug, Release or Checked.
                                  Checked is exclusive to the CLR subset. It is the same as Debug, except code is
                                  compiled with optimizations enabled.
                                  [Default: Debug]
  --help (-h)                     Print help and exit.
  --librariesConfiguration (-lc)  Libraries build configuration: Debug or Release.
                                  [Default: Debug]
  --os                            Target operating system: Windows_NT, Linux, FreeBSD, OSX, tvOS, iOS, Android,
                                  Browser, NetBSD or SunOS.
                                  [Default: Your machine's OS.]
  --projects <value>              Project or solution file(s) to build.
  --runtimeConfiguration (-rc)    Runtime build configuration: Debug, Release or Checked.
                                  Checked is exclusive to the CLR runtime. It is the same as Debug, except code is
                                  compiled with optimizations enabled.
                                  [Default: Debug]
  --subset (-s)                   Build a subset, print available subsets with -subset help.
                                 '--subset' can be omitted if the subset is given as the first argument.
                                  [Default: Builds the entire repo.]
  --verbosity (-v)                MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic].
                                  [Default: Minimal]

Actions (defaults to --restore --build):
  --build (-b)               Build all source projects.
                             This assumes --restore has been run already.
  --clean                    Clean the solution.
  --pack                     Package build outputs into NuGet packages.
  --publish                  Publish artifacts (e.g. symbols).
                             This assumes --build has been run already.
  --rebuild                  Rebuild all source projects.
  --restore (-r)             Restore dependencies.
  --sign                     Sign build outputs.
  --test (-t)                Incrementally builds and runs tests.
                             Use in conjuction with --testnobuild to only run tests.

Libraries settings:
  --allconfigurations        Build packages for all build configurations.
  --coverage                 Collect code coverage when testing.
  --framework (-f)           Build framework: net5.0 or net472.
                             [Default: net5.0]
  --testnobuild              Skip building tests when invoking -test.
  --testscope                Test scope, allowed values: innerloop, outerloop, all.

Native build settings:
  --clang                    Optional argument to build using clang in PATH (default).
  --clangx                   Optional argument to build using clang version x (used for Clang 7 and newer).
  --clangx.y                 Optional argument to build using clang version x.y (used for Clang 6 and older).
  --cmakeargs                User-settable additional arguments passed to CMake.
  --gcc                      Optional argument to build using gcc in PATH (default).
  --gccx.y                   Optional argument to build using gcc version x.y.

Command line arguments starting with '/p:' are passed through to MSBuild.
Arguments can also be passed in with a single hyphen.

Here are some quick examples. These assume you are on a Linux x64 machine:

* Build CoreCLR for Linux x64 on Release configuration:
./build.sh clr -c release

* Build Debug libraries with a Release runtime for Linux x64.
./build.sh clr+libs -rc release

* Build Release libraries and their tests with a Checked runtime for Linux x64, and run the tests.
./build.sh clr+libs+libs.tests -rc checked -lc release -test

* Build CoreCLR for Linux x64 on Debug configuration using Clang 9.
./build.sh clr -clang9

* Build CoreCLR for Linux x64 on Debug configuration using GCC 8.4.
./build.sh clr -gcc8.4

* Cross-compile CoreCLR runtime for Linux ARM64 on Release configuration.
./build.sh clr.runtime -arch arm64 -c release -cross

However, for this example, you need to already have ROOTFS_DIR set up.
Further information on this can be found here:
https://github.com/dotnet/runtime/blob/master/docs/workflow/building/coreclr/linux-instructions.md

* Build Mono runtime for Linux x64 on Release configuration.
./build.sh mono -c release

It's important to mention that to build Mono for the first time,
you need to build the CLR and Libs subsets beforehand.
This is done automatically if a full build is performed at first.

For more general information, check out https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md

BruceForstall · 2020-06-05T19:38:36Z

eng/build.ps1

+  Write-Host "                                 [Default: Your machine's architecture.]"
+  Write-Host "  -binaryLog (-bl)               Output binary log."
+  Write-Host "  -configuration (-c)            Build configuration: Debug, Release or Checked."
+  Write-Host "                                 Checked is exclusive to the CLR subset. Use it for optimized code with asserts and"


This is incorrect: Checked flavor also sets the DEBUG macro. The only difference between Debug and Checked flavors is that the Checked flavor is compiled with compiler and linker optimizations, so it runs faster. I would write (here and elsewhere):

Checked is exclusive to the CLR subset. It is the same as Debug, but is compiled with optimizations enabled so runs faster.

The downside of being compiled with optimizations is it is sometimes harder to debug. But we don't need to write a novel here :-)

Oh I see. I had misunderstood how the DEBUG macro is enabled. Thanks for the clarification!

Fixed it. I also updated the previous comment with the latest output.

danmoseley · 2020-06-05T22:02:21Z

@stephentoub any other feedback on the output?

safern · 2020-06-05T22:32:01Z

eng/build.sh

+      showSubsetHelp
+      exit 0
+    fi
+
    arguments="$arguments /p:Subset=$1"


Now that you're touching this, should /p:Subset=$1 be /p:Subset=$opt? don't know if that'll make a difference, but it is weird to check if $opt matches the regex, if opt is == help and then use $1.

I think you're right and using $opt would make the code look cleaner. Just before making this change, the arguments for MSBuild are not case-sensitive right? In the sense that /p:Subset=CLR would work just fine like /p:Subset=clr? Just making sure :)

IIRC it is case sensitive in Unix, but we lowercase the subsets before processing them:

runtime/eng/Subsets.props

Line 46 in 8f1c72d

<_subset Condition="'$(Subset)' != ''">+$(Subset.ToLowerInvariant())+</_subset>

danmoseley

LGTM except for --clangx.y seems like it should be --clangx

Thanks for improving this!

BruceForstall · 2020-06-05T22:40:33Z

except for --clangx.y seems like it should be --clangx

Perhaps more clearly, --clangVERSION where VERSION is 9, 8, ... because it used to be "x.y" and now is just "x" (no minor version)?

safern

Thanks for cleaning this up!

janvorli · 2020-06-05T22:57:23Z

LGTM except for --clangx.y seems like it should be --clangx

It depends on the clang version. Versions <= 6.0 use the x.y versioning schema, versions >=7 use just single number. LLVM has changed the versioning this way.

ivdiazsa · 2020-06-05T23:05:20Z

LGTM except for --clangx.y seems like it should be --clangx

It depends on the clang version. Versions <= 6.0 use the x.y versioning schema, versions >=7 use just single number. LLVM has changed the versioning this way.

For the sake of keeping "good historical track", I believe we should leave both in this case. I'll correct --clang to --clangx nonetheless (or would --clangVERSION be preferred?). Now that we are touching this, should --gcc be changed to --gccx as well? Or do GCC versions always use the x.y notation? @janvorli @danmosemsft

danmoseley · 2020-06-05T23:53:26Z

do GCC versions always use the x.y notation?

don't know

janvorli · 2020-06-06T08:35:17Z

Yes, gcc version is always x.y

ivdiazsa added 4 commits June 2, 2020 19:05

Sorted the build scripts' help alphabetically, and made the subset error

92bba0a

message softer and without metrics.

Added simple examples to the scripts' helps.

2a7f616

Moved short flags to the flag column and rephrased a bit.

9299ca7

Removed remains of old code.

efd1954

ivdiazsa requested review from danmoseley, janvorli, BruceForstall and trylek June 4, 2020 01:05

Dotnet-GitSync-Bot added the area-Infrastructure label Jun 4, 2020

safern requested a review from a team June 4, 2020 01:13

safern reviewed Jun 4, 2020

View reviewed changes

jaredpar mentioned this pull request Jun 4, 2020

OSX machines are de-provisioned during CI / PR runs leading to failures #34472

Closed

BruceForstall approved these changes Jun 4, 2020

View reviewed changes

danmoseley reviewed Jun 4, 2020

View reviewed changes

ViktorHofer reviewed Jun 4, 2020

View reviewed changes

danmoseley reviewed Jun 4, 2020

View reviewed changes

ViktorHofer reviewed Jun 4, 2020

View reviewed changes

danmoseley reviewed Jun 4, 2020

View reviewed changes

ivdiazsa added 2 commits June 4, 2020 18:45

Added more examples and rephrased some things for better clarity.

42280b5

Added gcc example to build.sh.

f1987b8

BruceForstall reviewed Jun 5, 2020

View reviewed changes

Corrected definition of the Checked configuration.

0601521

BruceForstall approved these changes Jun 5, 2020

View reviewed changes

stephentoub approved these changes Jun 5, 2020

View reviewed changes

safern reviewed Jun 5, 2020

View reviewed changes

danmoseley approved these changes Jun 5, 2020

View reviewed changes

safern approved these changes Jun 5, 2020

View reviewed changes

Added --clangx to build.sh

eb5fa42

ivdiazsa merged commit da97f17 into dotnet:master Jun 9, 2020

ghost locked as resolved and limited conversation to collaborators Dec 8, 2020

ivdiazsa deleted the show_time branch March 1, 2021 21:51

ivdiazsa restored the show_time branch March 1, 2021 21:51

ivdiazsa deleted the show_time branch March 1, 2021 21:51

	echo "Build Debug libraries with a Release runtime."
	echo "Build Debug libraries for Linux with a Release runtime."

		$argumentsForHelp = "-restore -build /p:subset=help /clp:nosummary"
		Invoke-Expression "& `"$PSScriptRoot/common/build.ps1`" $argumentsForHelp"

		argumentsForHelp="-restore -build /p:subset=help /clp:nosummary"
		"$scriptroot/common/build.sh" $argumentsForHelp

	Write-Host ".\build.cmd -subset clr+libs -os Windows_NT -arch x64 -runtimeConfiguration Release"
	Write-Host ".\build.cmd clr+libs -rc Release"

	Write-Host " -vs Open the solution with VS using the locally acquired SDK. Path or solution name."
	Write-Host " -vs Open the solution with Visual Studio using the locally acquired SDK. Path or solution name."

	Write-Host " -framework (-f) Build framework: net5.0 or net472."
	Write-Host " -framework (-f) Target framework: net5.0 or net472. Defaults to net5.0."

	Write-Host " -build (-b) Build all source projects."
	Write-Host " -build (-b) Build all source projects. (Assumes you already ran -restore)"

	Write-Host " -publish Publish artifacts (e.g. symbols)."
	Write-Host " -publish Publish artifacts (e.g. symbols - assumes you already ran -build)."

	# Check if an action is passed in
	$actions = "b","build","r","restore","rebuild","sign","testnobuild","publish","clean"
	$actionPassedIn = @(Compare-Object -ReferenceObject @($PSBoundParameters.Keys) -DifferenceObject $actions -ExcludeDifferent -IncludeEqual).Length -ne 0
	if ($null -ne $properties -and $actionPassedIn -ne $true) {
	$actionPassedIn = @(Compare-Object -ReferenceObject $properties -DifferenceObject $actions.ForEach({ "-" + $_ }) -ExcludeDifferent -IncludeEqual).Length -ne 0
	}

Make build scripts' -help information more helpful #37377

Make build scripts' -help information more helpful #37377

Uh oh!

Conversation

ivdiazsa commented Jun 4, 2020

Uh oh!

ghost commented Jun 4, 2020

Uh oh!

danmoseley commented Jun 4, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ivdiazsa commented Jun 4, 2020

Uh oh!

BruceForstall left a comment

Choose a reason for hiding this comment

Uh oh!

danmoseley commented Jun 4, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ivdiazsa Jun 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danmoseley Jun 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

ivdiazsa Jun 4, 2020 •

edited

Loading

danmoseley Jun 4, 2020 •

edited

Loading

ViktorHofer commented Jun 4, 2020 •

edited by safern

Loading

ivdiazsa commented Jun 4, 2020 •

edited

Loading

BruceForstall commented Jun 4, 2020 •

edited by safern

Loading

ivdiazsa commented Jun 5, 2020 •

edited

Loading