Skip to content

Make build scripts' -help information more helpful #37377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 9, 2020
Merged

Make build scripts' -help information more helpful #37377

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
92bba0a
Sorted the build scripts' help alphabetically, and made the subset error
ivdiazsa Jun 3, 2020
2a7f616
Added simple examples to the scripts' helps.
ivdiazsa Jun 3, 2020
9299ca7
Moved short flags to the flag column and rephrased a bit.
ivdiazsa Jun 4, 2020
efd1954
Removed remains of old code.
ivdiazsa Jun 4, 2020
42280b5
Added more examples and rephrased some things for better clarity.
ivdiazsa Jun 5, 2020
f1987b8
Added gcc example to build.sh.
ivdiazsa Jun 5, 2020
0601521
Corrected definition of the Checked configuration.
ivdiazsa Jun 5, 2020
eb5fa42
Added --clangx to build.sh
ivdiazsa Jun 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/SubsetValidation.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
<Error Text="Subset not recognized: @(InvalidSpecifiedSubsetName, ' ')"
Condition="'$(UserRequestedHelp)' != 'true'" />

<Error Text="Choose a subset to use, or do not specify a subset to perform the full build."
<Error Text="This is not an error. These are the available subsets. You can choose one or none at all for a full build."
Condition="'$(UserRequestedHelp)' == 'true'" />
</Target>

Expand Down
100 changes: 75 additions & 25 deletions eng/build.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,46 +20,96 @@ Param(

function Get-Help() {
Write-Host "Common settings:"
Write-Host " -subset Build a subset, print available subsets with -subset help (short: -s)"
Write-Host " -vs Open the solution with VS using the locally acquired SDK. Path or solution name (ie -vs Microsoft.CSharp)"
Write-Host " -os Build operating system: Windows_NT, Linux, OSX, or Browser"
Write-Host " -arch Build platform: x86, x64, arm, arm64, or wasm (short: -a). Pass a comma-separated list to build for multiple architectures."
Write-Host " -configuration Build configuration: Debug, Release or [CoreCLR]Checked (short: -c). Pass a comma-separated list to build for multiple configurations"
Write-Host " -runtimeConfiguration Runtime build configuration: Debug, Release or [CoreCLR]Checked (short: -rc)"
Write-Host " -librariesConfiguration Libraries build configuration: Debug or Release (short: -lc)"
Write-Host " -verbosity MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
Write-Host " -binaryLog Output binary log (short: -bl)"
Write-Host " -help Print help and exit (short: -h)"
Write-Host " -arch (-a) Target platform: x86, x64, arm, arm64, or wasm."
Write-Host " Pass a comma-separated list to build for multiple architectures."
Write-Host " [Default: Your machine's architecture.]"
Write-Host " -binaryLog (-bl) Output binary log."
Write-Host " -configuration (-c) Build configuration: Debug, Release or Checked."
Write-Host " Checked is exclusive to the CLR subset. It is the same as Debug, except code is"
Write-Host " compiled with optimizations enabled."
Write-Host " Pass a comma-separated list to build for multiple configurations."
Write-Host " [Default: Debug]"
Write-Host " -help (-h) Print help and exit."
Write-Host " -librariesConfiguration (-lc) Libraries build configuration: Debug or Release."
Write-Host " [Default: Debug]"
Write-Host " -os Target operating system: Windows_NT, Linux, OSX, or Browser."
Write-Host " [Default: Your machine's OS.]"
Write-Host " -runtimeConfiguration (-rc) Runtime build configuration: Debug, Release or Checked."
Write-Host " Checked is exclusive to the CLR runtime. It is the same as Debug, except code is"
Write-Host " compiled with optimizations enabled."
Write-Host " [Default: Debug]"
Write-Host " -subset (-s) Build a subset, print available subsets with -subset help."
Write-Host " '-subset' can be omitted if the subset is given as the first argument."
Write-Host " [Default: Builds the entire repo.]"
Write-Host " -verbosity (-v) MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic]."
Write-Host " [Default: Minimal]"
Write-Host " -vs Open the solution with Visual Studio using the locally acquired SDK."
Write-Host " Path or any project or solution name is accepted."
Write-Host " (Example: -vs Microsoft.CSharp)"
Write-Host ""

Write-Host "Actions (defaults to -restore -build):"
Write-Host " -restore Restore dependencies"
Write-Host " -build Build all source projects (short: -b)"
Write-Host " -rebuild Rebuild all source projects"
Write-Host " -test Build and run tests (short: -t)"
Write-Host " -pack Package build outputs into NuGet packages"
Write-Host " -sign Sign build outputs"
Write-Host " -publish Publish artifacts (e.g. symbols)"
Write-Host " -clean Clean the solution"
Write-Host " -build (-b) Build all source projects."
Copy link
Member

@danmoseley danmoseley Jun 4, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Write-Host " -build (-b) Build all source projects."
Write-Host " -build (-b) Build all source projects. (Assumes you already ran -restore)"

I made this mistake in the past

Write-Host " This assumes -restore has been run already."
Write-Host " -clean Clean the solution."
Write-Host " -pack Package build outputs into NuGet packages."
Write-Host " -publish Publish artifacts (e.g. symbols)."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Write-Host " -publish Publish artifacts (e.g. symbols)."
Write-Host " -publish Publish artifacts (e.g. symbols - assumes you already ran -build)."

Write-Host " This assumes -build has been run already."
Write-Host " -rebuild Rebuild all source projects."
Write-Host " -restore Restore dependencies."
Write-Host " -sign Sign build outputs."
Write-Host " -test (-t) Incrementally builds and runs tests."
Write-Host " Use in conjuction with -testnobuild to only run tests."
Write-Host ""

Write-Host "Libraries settings:"
Write-Host " -framework Build framework: net5.0 or net472 (short: -f)"
Write-Host " -coverage Collect code coverage when testing"
Write-Host " -testscope Scope tests, allowed values: innerloop, outerloop, all"
Write-Host " -testnobuild Skip building tests when invoking -test"
Write-Host " -allconfigurations Build packages for all build configurations"
Write-Host " -allconfigurations Build packages for all build configurations."
Write-Host " -coverage Collect code coverage when testing."
Write-Host " -framework (-f) Build framework: net5.0 or net472."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Write-Host " -framework (-f) Build framework: net5.0 or net472."
Write-Host " -framework (-f) Target framework: net5.0 or net472. Defaults to net5.0."

Write-Host " [Default: net5.0]"
Write-Host " -testnobuild Skip building tests when invoking -test."
Write-Host " -testscope Scope tests, allowed values: innerloop, outerloop, all."
Write-Host ""

Write-Host "Command-line arguments not listed above are passed thru to msbuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous (e.g. -con for configuration, -t for test, etc.)."
Write-Host "Command-line arguments not listed above are passed through to MSBuild."
Write-Host "The above arguments can be shortened as much as to be unambiguous."
Write-Host "(Example: -con for configuration, -t for test, etc.)."
Write-Host ""

Write-Host "Here are some quick examples. These assume you are on a Windows x64 machine:"
Write-Host ""
Write-Host "* Build CoreCLR for Windows x64 on Release configuration:"
Write-Host ".\build.cmd clr -c release"
Write-Host ""
Write-Host "* Cross-compile CoreCLR runtime for Windows ARM64 on Release configuration."
Write-Host ".\build.cmd clr.runtime -arch arm64 -c release"
Write-Host ""
Write-Host "* Build Debug libraries with a Release runtime for Windows x64."
Write-Host ".\build.cmd clr+libs -rc release"
Write-Host ""
Write-Host "* Build Release libraries and their tests with a Checked runtime for Windows x64, and run the tests."
Write-Host ".\build.cmd clr+libs+libs.tests -rc checked -lc release -test"
Write-Host ""
Write-Host "* Build Mono runtime for Windows x64 on Release configuration."
Write-Host ".\build.cmd mono -c release"
Write-Host ""
Write-Host "It's important to mention that to build Mono for the first time,"
Write-Host "you need to build the CLR and Libs subsets beforehand."
Write-Host "This is done automatically if a full build is performed at first."
Write-Host ""
Write-Host "For more information, check out https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md"
}

if ($help -or (($null -ne $properties) -and ($properties.Contains('/help') -or $properties.Contains('/?')))) {
Get-Help
exit 0
}

if ($subset -eq 'help') {
Invoke-Expression "& `"$PSScriptRoot/common/build.ps1`" -restore -build /p:subset=help /clp:nosummary"
exit 0
}

if ($vs) {
. $PSScriptRoot\common\tools.ps1

Expand Down
128 changes: 98 additions & 30 deletions eng/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,47 +17,100 @@ scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
usage()
{
echo "Common settings:"
echo " --subset Build a subset, print available subsets with -subset help (short: -s)"
echo " --os Build operating system: Windows_NT, Linux, FreeBSD, OSX, tvOS, iOS, Android, Browser, NetBSD or SunOS"
echo " --arch Build platform: x86, x64, arm, armel, arm64 or wasm"
echo " --configuration Build configuration: Debug, Release or [CoreCLR]Checked (short: -c)"
echo " --runtimeConfiguration Runtime build configuration: Debug, Release or [CoreCLR]Checked (short: -rc)"
echo " --librariesConfiguration Libraries build configuration: Debug or Release (short: -lc)"
echo " --projects <value> Project or solution file(s) to build"
echo " --verbosity MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)"
echo " --binaryLog Output binary log (short: -bl)"
echo " --cross Optional argument to signify cross compilation"
echo " --help Print help and exit (short: -h)"
echo " --arch Target platform: x86, x64, arm, armel, arm64 or wasm."
echo " [Default: Your machine's architecture.]"
echo " --binaryLog (-bl) Output binary log."
echo " --cross Optional argument to signify cross compilation."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meaning what -- how do i use it, when do I need to pass it? If I build libraries with --os Linux, do i need --cross as well?

echo " --configuration (-c) Build configuration: Debug, Release or Checked."
echo " Checked is exclusive to the CLR subset. It is the same as Debug, except code is"
echo " compiled with optimizations enabled."
echo " [Default: Debug]"
echo " --help (-h) Print help and exit."
echo " --librariesConfiguration (-lc) Libraries build configuration: Debug or Release."
echo " [Default: Debug]"
echo " --os Target operating system: Windows_NT, Linux, FreeBSD, OSX, tvOS, iOS, Android,"
echo " Browser, NetBSD or SunOS."
echo " [Default: Your machine's OS.]"
echo " --projects <value> Project or solution file(s) to build."
echo " --runtimeConfiguration (-rc) Runtime build configuration: Debug, Release or Checked."
echo " Checked is exclusive to the CLR runtime. It is the same as Debug, except code is"
echo " compiled with optimizations enabled."
echo " [Default: Debug]"
echo " --subset (-s) Build a subset, print available subsets with -subset help."
echo " '--subset' can be omitted if the subset is given as the first argument."
echo " [Default: Builds the entire repo.]"
echo " --verbosity (-v) MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic]."
echo " [Default: Minimal]"
echo ""

echo "Actions (defaults to --restore --build):"
echo " --restore Restore dependencies (short: -r)"
echo " --build Build all source projects (short: -b)"
echo " --rebuild Rebuild all source projects"
echo " --test Build and run tests (short: -t)"
echo " --pack Package build outputs into NuGet packages"
echo " --sign Sign build outputs"
echo " --publish Publish artifacts (e.g. symbols)"
echo " --clean Clean the solution"
echo " --build (-b) Build all source projects."
echo " This assumes --restore has been run already."
echo " --clean Clean the solution."
echo " --pack Package build outputs into NuGet packages."
echo " --publish Publish artifacts (e.g. symbols)."
echo " This assumes --build has been run already."
echo " --rebuild Rebuild all source projects."
echo " --restore (-r) Restore dependencies."
echo " --sign Sign build outputs."
echo " --test (-t) Incrementally builds and runs tests."
echo " Use in conjuction with --testnobuild to only run tests."
echo ""

echo "Libraries settings:"
echo " --framework Build framework: net5.0 or net472 (short: -f)"
echo " --coverage Collect code coverage when testing"
echo " --testscope Test scope, allowed values: innerloop, outerloop, all"
echo " --testnobuild Skip building tests when invoking -test"
echo " --allconfigurations Build packages for all build configurations"
echo " --allconfigurations Build packages for all build configurations."
echo " --coverage Collect code coverage when testing."
echo " --framework (-f) Build framework: net5.0 or net472."
echo " [Default: net5.0]"
echo " --testnobuild Skip building tests when invoking -test."
echo " --testscope Test scope, allowed values: innerloop, outerloop, all."
echo ""

echo "Native build settings:"
echo " --clang Optional argument to build using clang in PATH (default)"
echo " --clangx.y Optional argument to build using clang version x.y"
echo " --clang Optional argument to build using clang in PATH (default)."
echo " --clangx Optional argument to build using clang version x (used for Clang 7 and newer)."
echo " --clangx.y Optional argument to build using clang version x.y (used for Clang 6 and older)."
echo " --cmakeargs User-settable additional arguments passed to CMake."
echo " --gcc Optional argument to build using gcc in PATH (default)"
echo " --gccx.y Optional argument to build using gcc version x.y"
echo " --gcc Optional argument to build using gcc in PATH (default)."
echo " --gccx.y Optional argument to build using gcc version x.y."
echo ""

echo "Command line arguments starting with '/p:' are passed through to MSBuild."
echo "Arguments can also be passed in with a single hyphen."
echo ""

echo "Here are some quick examples. These assume you are on a Linux x64 machine:"
echo ""
echo "* Build CoreCLR for Linux x64 on Release configuration:"
echo "./build.sh clr -c release"
echo ""
echo "* Build Debug libraries with a Release runtime for Linux x64."
echo "./build.sh clr+libs -rc release"
echo ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding build -subset clr+libs -runtimeConfiguration Release (it's what we suggest for library work in docs\workflow\building\libraries\README.md)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And note that that one picks the same target platform you are building on

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW the shortest form (that most of us use) is build clr+libs -rc Release.

echo "* Build Release libraries and their tests with a Checked runtime for Linux x64, and run the tests."
echo "./build.sh clr+libs+libs.tests -rc checked -lc release -test"
echo ""
echo "* Build CoreCLR for Linux x64 on Debug configuration using Clang 9."
echo "./build.sh clr -clang9"
echo ""
echo "* Build CoreCLR for Linux x64 on Debug configuration using GCC 8.4."
echo "./build.sh clr -gcc8.4"
echo ""
echo "* Cross-compile CoreCLR runtime for Linux ARM64 on Release configuration."
echo "./build.sh clr.runtime -arch arm64 -c release -cross"
echo ""
echo "However, for this example, you need to already have ROOTFS_DIR set up."
echo "Further information on this can be found here:"
echo "https://github.com/dotnet/runtime/blob/master/docs/workflow/building/coreclr/linux-instructions.md"
echo ""
echo "* Build Mono runtime for Linux x64 on Release configuration."
echo "./build.sh mono -c release"
echo ""
echo "It's important to mention that to build Mono for the first time,"
echo "you need to build the CLR and Libs subsets beforehand."
echo "This is done automatically if a full build is performed at first."
echo ""
echo "For more general information, check out https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md"
}

initDistroRid()
Expand All @@ -79,6 +132,11 @@ initDistroRid()
initDistroRidGlobal ${targetOs} ${buildArch} ${isPortableBuild} ${passedRootfsDir}
}

showSubsetHelp()
{
"$scriptroot/common/build.sh" "-restore" "-build" "/p:Subset=help" "/clp:nosummary"
}

arguments=''
cmakeargs=''
extraargs=''
Expand All @@ -95,6 +153,11 @@ while [[ $# > 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"

if [[ $firstArgumentChecked -eq 0 && $opt =~ ^[a-zA-Z.+]+$ ]]; then
if [ $opt == "help" ]; then
showSubsetHelp
exit 0
fi

arguments="$arguments /p:Subset=$1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that you're touching this, should /p:Subset=$1 be /p:Subset=$opt? don't know if that'll make a difference, but it is weird to check if $opt matches the regex, if opt is == help and then use $1.

Copy link
Contributor Author

@ivdiazsa ivdiazsa Jun 5, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right and using $opt would make the code look cleaner. Just before making this change, the arguments for MSBuild are not case-sensitive right? In the sense that /p:Subset=CLR would work just fine like /p:Subset=clr? Just making sure :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC it is case sensitive in Unix, but we lowercase the subsets before processing them:

runtime/eng/Subsets.props

Line 46 in 8f1c72d

<_subset Condition="'$(Subset)' != ''">+$(Subset.ToLowerInvariant())+</_subset>

shift 1
continue
Expand All @@ -110,9 +173,14 @@ while [[ $# > 0 ]]; do

-subset|-s)
if [ -z ${2+x} ]; then
arguments="$arguments /p:Subset=help"
shift 1
showSubsetHelp
exit 0
else
passedSubset="$(echo "$2" | awk '{print tolower($0)}')"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also have detection on the first argument to parse it as a subset: https://github.com/dotnet/runtime/pull/37377/files#diff-75d32cbca8eddea951ce0484aa996d8fR130

So we also need to consider the case where people passes down:

.\build.sh help

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tested it. It works and shows the new subset message, but it shows the metrics. Let me add a handler for that case as well.

if [ $passedSubset == "help" ]; then
showSubsetHelp
exit 0
fi
arguments="$arguments /p:Subset=$2"
shift 2
fi
Expand Down