Redesign the subset feature (dotnet#34663)

* Redesign subset feature and remove subsetcategory

Implement proposal from dotnet#34403.
Remove subsetcategory and allow all subsets to specified via the -subset
switch.

Author: ViktorHofer

docs/workflow/README.md

@@ -38,7 +38,7 @@ build -h
 On Unix like systems, arguments can be passed in with a single `-` or double hyphen `--`.
 
 The repository currently consists of different major parts: the runtimes, the libraries, and the installer.
-To build just one part you use the root build script (build.cmd/sh), and you add the `-subsetCategory` flag.
+To build just one part you use the root build script (build.cmd/sh), and you add the `-subset` flag.
 
 ## Configurations
 

docs/workflow/building/coreclr/README.md

@@ -1,25 +1,25 @@
 # Building
 
-To build just CoreCLR, use the `-subsetCategory` flag to the `build.sh` (or `build.cmd`) script at the repo root:
+To build just CoreCLR, use the `-subset` flag to the `build.sh` (or `build.cmd`) script at the repo root:
 
 For Linux:
 ```
-./build.sh -subsetCategory coreclr
+./build.sh -subset clr
 ```
 
 For Windows:
 ```
-build.cmd -subsetCategory coreclr
+build.cmd -subset clr
 ```
 
 By default, build generates a 'debug' build type, that includes asserts and is easier for some people to debug. If you want to make performance measurements, or just want tests to execute more quickly, you can also build the 'release' version (which does not have these checks) by adding the flag `-configuration release` (or `-c release`), for example:
 ```
-./build.sh -subsetCategory coreclr -configuration release
+./build.sh -subset clr -configuration release
 ```
 
 CoreCLR also supports a 'checked' build type which has asserts enabled like 'debug', but is built with the native compiler optimizer enabled, so it runs much faster. This is the usual mode used for running tests in the CI system. You can build that using, for example:
 ```
-./build.sh -subsetCategory coreclr -configuration checked
+./build.sh -subset clr -configuration checked
 ```
 
 This will produce outputs as follows:

docs/workflow/building/coreclr/cross-building.md

@@ -13,7 +13,7 @@ Cross compiling CoreCLR
 
 Build using "arm" as the architecture. For example:
 
-    C:\runtime> build.cmd -subsetCategory CoreClr -subset Runtime -arch arm -c debug
+    C:\runtime> build.cmd -subset clr.runtime -arch arm -c debug
 
 -or-
 
@@ -101,7 +101,7 @@ Cross compiling CoreCLR
 -----------------------
 `ROOTFS_DIR` must be set when running `build-runtime.sh`.
 
-    ~/runtime/ $ ROOTFS_DIR=/home/arm ./build.sh --subsetCategory CoreClr --subset Runtime --arch arm -c debug -v verbose --cross
+    ~/runtime/ $ ROOTFS_DIR=/home/arm ./build.sh --subset clr.runtime --arch arm -c debug -v verbose --cross
 
 -or-
 
@@ -144,7 +144,7 @@ The following instructions assume you are on a Linux machine such as Ubuntu 14.0
 To build System.Private.CoreLib for Linux, run the following command:
 
 ```
-    lgs@ubuntu ~/git/runtime/ $ ./build.sh --subsetCategory CoreClr --subset CoreLib-NativeCoreLib --arch arm -c debug -v verbose
+    lgs@ubuntu ~/git/runtime/ $ ./build.sh --subset clr.corelib+clr.nativecorelib --arch arm -c debug -v verbose
 ```
 
 The output is at `artifacts/bin/coreclr/<TargetOS>.arm.Debug/IL/System.Private.CoreLib.dll`.

docs/workflow/building/coreclr/linux-instructions.md

@@ -127,7 +127,7 @@ Build the Runtime and System.Private.CoreLib
 To build the runtime on Linux, run build.sh to build the CoreCLR subset category of the runtime:
 
 ```
-./build.sh -subsetCategory coreclr
+./build.sh -subset clr
 ```
 
 After the build is completed, there should some files placed in `artifacts/bin/coreclr/Linux.x64.Debug`.  The ones we are most interested in are:

docs/workflow/building/coreclr/osx-instructions.md

@@ -46,7 +46,7 @@ Build the Runtime and System.Private.CoreLib
 To Build CoreCLR, run build.sh to build the CoreCLR subset category of the runtime:
 
 ```
-./build.sh -subsetCategory coreclr
+./build.sh -subset clr
 ```
 
 After the build has completed, there should some files placed in `artifacts/bin/coreclr/OSX.x64.Debug`. The ones we are interested in are:

docs/workflow/building/libraries/README.md

@@ -9,7 +9,7 @@ Here is one example of a daily workflow for a developer working mainly on the li
 git clean -xdf
 git pull upstream master & git push origin master
 :: Build Debug libraries on top of Release runtime:
-build -subsetCategory coreclr-libraries -runtimeConfiguration Release
+build -subset clr+libs -runtimeConfiguration Release
 :: The above you may only perform once in a day, or when you pull down significant new changes.
 
 :: If you use Visual Studio, you might open System.Text.RegularExpressions.sln here.
@@ -33,7 +33,7 @@ The instructions for Linux and macOS are essentially the same:
 git clean -xdf
 git pull upstream master & git push origin master
 # Build Debug libraries on top of Release runtime:
-./build.sh -subsetCategory coreclr-libraries -runtimeconfiguration Release
+./build.sh -subset clr+libs -runtimeconfiguration Release
 # The above you may only perform once in a day, or when you pull down significant new changes.
 
 # Switch to working on a given library (RegularExpressions in this case)
@@ -83,33 +83,28 @@ For more details on the build settings see [project-guidelines](../../../coding-
 
 If you invoke the `build` script without any actions, the default action chain `-restore -build` is executed.
 
-By default the `build` script only builds the product libraries and none of the tests. If you want to include tests, you want to add the subset `-subset libtests`. If you want to run the tests you want to use the `-test` action instead of the `-build`, e.g. `build.cmd/sh -subsetcategory libraries -test`. To specify just the libraries, use `-subcategory libraries`.
+By default the `build` script only builds the product libraries and none of the tests. If you want to include tests, you want to add the subset `-subset libtests`. If you want to run the tests you want to use the `-test` action instead of the `-build`, e.g. `build.cmd/sh -subset libs.tests -test`. To specify just the libraries, use `-subset libs`.
 
 **Examples**
 - Building in release mode for platform x64 (restore and build are implicit here as no actions are passed in)
 ```bash
-./build.sh -subsetCategory libraries -c Release -arch x64
+./build.sh -subset libs -c Release -arch x64
 ```
 
 - Building the src assemblies and build and run tests (running all tests takes a considerable amount of time!)
 ```bash
-./build.sh -subsetCategory libraries -restore -build -test
+./build.sh -subset libs -test
 ```
 
 - Building for different target frameworks (restore and build are implicit again as no action is passed in)
 ```bash
-./build.sh -subsetCategory libraries -framework netcoreapp5.0
-./build.sh -subsetCategory libraries -framework net472
-```
-
-- Build only managed components and skip the native build
-```bash
-./build.sh -subsetCategory libraries /p:BuildNative=false
+./build.sh -subset libs -framework netcoreapp5.0
+./build.sh -subset libs -framework net472
 ```
 
 - Clean the entire solution
 ```bash
-./build.sh -subsetCategory libraries -clean
+./build.sh -clean
 ```
 
 For Windows, replace `./build.sh` with `build.cmd`.
@@ -141,17 +136,17 @@ Similar to building the entire repo with `build.cmd` or `build.sh` in the root y
 - Build all projects for a given library (e.g.: System.Collections) including running the tests
 
 ```bash
- ./build.sh -subsetCategory libraries src/libraries/System.Collections
+ ./build.sh -projects src/libraries/*/System.Collections.sln
 ```
 
 - Build just the tests for a library project
 ```bash
- ./build.sh -subsetCategory libraries src/libraries/System.Collections/tests
+ ./build.sh -projects src/libraries/System.Collections/tests/*.csproj
 ```
 
 - All the options listed above like framework and configuration are also supported (note they must be after the directory)
 ```bash
- ./build.sh -subsetCategory libraries System.Collections -f net472 -c Release
+ ./build.sh -projects src/libraries/*/System.Collections.sln -f net472 -c Release
 ```
 
 As `dotnet build` works on both Unix and Windows and calls the restore target implicitly, we will use it throughout this guide.
@@ -185,24 +180,24 @@ dotnet build -c Release System.Net.NetworkInformation.csproj
 By default the libraries will attempt to build using the CoreCLR version of `System.Private.CoreLib.dll`. In order to build against the Mono version you need to use the `/p:RuntimeFlavor=Mono` argument.
 
 ```
-.\build.cmd -subsetCategory libraries /p:RuntimeFlavor=Mono
+.\build.cmd -subset libs /p:RuntimeFlavor=Mono
 ```
 
 ### Building all for other OSes
 
 By default, building from the root will only build the libraries for the OS you are running on. One can
-build for another OS by specifying `./build.sh -subsetCategory libraries -os [value]`.
+build for another OS by specifying `./build.sh -subset libs -os [value]`.
 
 Note that you cannot generally build native components for another OS but you can for managed components so if you need to do that you can do it at the individual project level or build all via passing `/p:BuildNative=false`.
 
 ### Building in Release or Debug
 
 By default, building from the root or within a project will build the libraries in Debug mode.
-One can build in Debug or Release mode from the root by doing `./build.sh -subsetCategory libraries  -c Release` or `./build.sh -subsetCategory libraries -c Debug`.
+One can build in Debug or Release mode from the root by doing `./build.sh -subset libs -c Release` or `./build.sh -subset libs -c Debug`.
 
 ### Building other Architectures
 
-One can build 32- or 64-bit binaries or for any architecture by specifying in the root `./build.sh -subsetCategory libraries -arch [value]` or in a project `/p:TargetArchitecture=[value]` after the `dotnet build` command.
+One can build 32- or 64-bit binaries or for any architecture by specifying in the root `./build.sh -subset libs -arch [value]` or in a project `/p:TargetArchitecture=[value]` after the `dotnet build` command.
 
 ## Working in Visual Studio
 

docs/workflow/building/mono/README.md

@@ -15,11 +15,11 @@ Note that the debug configuration is the default option. It generates a 'debug'
 Once you've built the whole runtime and assuming you want to work with just mono, you want to use the following command:
 
 ```bash
-./build.sh --subsetCategory mono
+./build.sh --subset mono
 ```
 or on Windows,
 ```bat
-build.cmd -subsetCategory mono
+build.cmd -subset mono
 ```
 When the build completes, product binaries will be dropped in the `artifacts\bin\mono\<OS>.<arch>.<flavor>` folder.
 
@@ -44,11 +44,11 @@ The build has a number of options that you can learn about using build -?.
 To generate nuget packages:
 
 ```bash
-./build.sh --subsetCategory mono -pack (with optional release configuration)
+./build.sh --subset mono -pack (with optional release configuration)
 ```
 or on Windows,
 ```bat
-build.cmd -subsetCategory mono -pack (with optional release configuration)
+build.cmd -subset mono -pack (with optional release configuration)
 ```
 
 The following packages will be created under `artifacts\packages\<configuration>\Shipping`:

docs/workflow/testing/libraries/testing.md

@@ -7,28 +7,28 @@ To build the tests and run them you can call the libraries build script.
 **Examples**
 - The following shows how to build only the tests but not run them:
 ```
-libraries -subset libtests
+build.cmd/sh -subset libs.tests
 ```
 
 - The following builds and runs all tests in release configuration:
 ```
-libraries -test -c Release
+build.cmd/sh -subset libs -test -c Release
 ```
 
 - The following example shows how to pass extra msbuild properties to ignore tests ignored in CI:
 ```
-libraries -test /p:WithoutCategories=IgnoreForCI
+build.cmd/sh -subset libs -test /p:WithoutCategories=IgnoreForCI
 ```
 
 Unless you specifiy `-testnobuild`, test assemblies are implicitly built when invoking the `Test` action.
 - The following shows how to only test the libraries without building them
 ```
-libraries -test -testnobuild
+build.cmd/sh -subset libs -test -testnobuild
 ```
 
 ## Running tests on the command line
 
-To build tests you need to specify the `test` subset when invoking build.cmd/sh: `libraries -subset libtests`.
+To build tests you need to specify the `test` subset when invoking build.cmd/sh: `build.cmd/sh -subset libs.tests`.
 
 The easiest (and recommended) way to build and run the tests for a specific library, is to invoke the `Test` target on that library:
 ```cmd

eng/SubsetValidation.targets

@@ -15,7 +15,7 @@
 
   <Target Name="FindInvalidSpecifiedSubsetNames">
     <ItemGroup>
-      <SpecifiedSubsetName Include="$([MSBuild]::Unescape($(Subset.Replace('-', ';'))))" />
+      <SpecifiedSubsetName Include="$([MSBuild]::Unescape($(Subset.Replace('+', ';').Replace('-', ';'))))" />
 
       <!-- MSBuild Exclude is case-insensitive, which matches intended behavior. -->
       <InvalidSpecifiedSubsetName Include="@(SpecifiedSubsetName)" Exclude="@(SubsetName)" />
@@ -29,9 +29,7 @@
   <Target Name="ReportValidSubsetList"
           Condition="'@(InvalidSpecifiedSubsetName)' != ''">
     <ItemGroup>
-      <SubsetName Text="- " />
-      <SubsetName Condition="'%(Category)' != ''" Text="%(Text)%(Category)|" />
-      <SubsetName Text="%(Text)%(Identity)" />
+      <SubsetName Text="- %(Identity)" />
       <SubsetName Text="%(Text) [only runs on demand]" Condition="'%(SubsetName.OnDemand)' == 'true'" />
       <SubsetName Text="%(Text)%0A    %(Description)" />
     </ItemGroup>