[Android] Setup `dotnet new android` performance runs

[matouskozak]

The goal is to setup a continuous lab measurement of startup and size for dotnet new android sample app. Requirements are to support Mono and CoreCLR (when available) runtimes. Additionally, to have a tight dev loop, we need appropriate cadence of runtime (SDK) and Android workload builds.

Proposal

Using VMR SDK builds (±daily cadence) and nightly Android workload from dotnet10 feed (https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet10/nuget/v3/index.json).

Stage 1 (daily SDK + latest Android workload release)

In the first stage we will setup a measurement for daily runtime builds obtained via VMR builds (https://github.com/dotnet/dotnet/blob/main/docs/builds-table.md) together with the latest released Android workload (not incremental builds). This can be achieved by the following steps.

1. Download dotnet-install script (https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script)
2. Install the latest VMR SDK build: ./dotnet-install.sh --channel "10.0.1xx-ub" --quality daily --install-dir sdk
3. Create a nuget.config with dotnet10 feed
   a. /sdk/dotnet new nugetconfig
   b. Add: <add key="dotnet10" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet10/nuget/v3/index.json" />
   c. Add "dotnet-android" feeds from https://github.com/dotnet/android/blob/main/NuGet.config
4. Install the Android workload: ./sdk/dotnet workload install android
5. Create the Android sample app: ./sdk/dotnet new android

The output of these steps as of 2025/03/07 is:

• Runtime version: 10.0.0-preview.3.25155.15 (commit: 666e9188f5e11f4a59708b5f18ff611aa950d965)
• Android workload version: 35.99.0-preview.1.140/10.0.100-preview.1 (commit: 6ec531d9418573d56a66192a59411f548f17844d)

Stage 2 (daily SDK + nightly Android workload builds)

In the second stage we improve the setup by adding nightly Android workload builds. This can be achieved by the following steps:

Option 1

1. Repeat steps 1.-3. from Stage 1
2. Setup workload to grap the latest manifests: ./sdk/dotnet workload config --update-mode manifests
3. Install the Android workload: ./sdk/dotnet workload install android
4. Create the Android sample app: ./sdk/dotnet new android

Option 2

1. Repeat steps 1.-4. from Stage 1
2. Create a rollback file with

{
    "microsoft.net.sdk.android": "35.99.0-ci.main.220/10.0.100-preview.3"
}

3. Update workload: ./sdk/dotnet workload update --from-rollback-file Rollback.json
4. Create the Android sample app: ./sdk/dotnet new android

The rollback files must be updated automatically, for example, by darc subscription. Steps TBD.

The output of these steps as of 2025/03/07 is:

• Runtime version: 10.0.0-preview.3.25155.15 (commit: 666e9188f5e11f4a59708b5f18ff611aa950d965)
• Android workload version: 35.99.0-ci.main.220/10.0.100-preview.3 (commit: 93b17c0f844c2ecd4b271ebac8ff3ea94c6fcc84)

How to determine corresponding commits

Runtime

TBD

Android

• Option A: sdk/packs/Microsoft.Android.Sdk.Darwin/35.99.0-ci.main.220/tools/Version.commit ->

dotnet/android/main/93b17c0

• Option B: sdk/packs/Microsoft.Android.Runtime.Mono.35.android-arm64/35.99.0-ci.main.220/Microsoft.Android.Runtime.Mono.35.android-arm64.nuspec ->

<repository type="git" url="https://github.com/dotnet/android" commit="93b17c0f844c2ecd4b271ebac8ff3ea94c6fcc84" />

[LoopedBard3]

A few questions about some of the parts:

• Does the dotnet-install channel have to be "10.0.1xx-ub" or should channel main and quality daily work? We already always use channel main and quality daily in the performance scenario runs.
• For the Nuget.config file, should we be able to use the top level one in the performance repo? We already keep that one up to date and it is known to be properly setup.

Thank you for laying this out!

[matouskozak]

• Does the dotnet-install channel have to be "10.0.1xx-ub" or should channel main and quality daily work? We already always use channel main and quality daily in the performance scenario runs.

We can use the default main channel as well. The "10.0.1xx-ub" channel was picked as it should contain VMR SDK builds. However, it is likely that these will become part of the default channel.

• For the Nuget.config file, should we be able to use the top level one in the performance repo? We already keep that one up to date and it is known to be properly setup.

Yes, I wasn't aware that there is already a Nuget.config prefilled with the latest feeds. I verified locally and it works well.