Add unified-build lite pipeline

[NikolaMilosavljevic]

Validation of vmr changes does not always require running all unified-build legs.

Having a lite pipeline could help improve throughput, and save build resources.

This could also enable us to validate legs that build tests, as those are now only enabled in public CI or PR.

Some concerns

Today we run jobs that build tests in public Validation jobs only: https://github.com/dotnet/sdk/blob/d833ffb8016b3937b6c13db926fc1acbac254dd7/eng/pipelines/templates/stages/vmr-build.yml#L354-L387

To avoid duplicating jobs in main pipeline yml, perhaps we should just update that condition to include internal/lite scenario and update the comment.

Not sure if both Linux and Windows jobs for building tests would be required, or just one of them.

[NikolaMilosavljevic]

@mmitche @ViktorHofer

[NikolaMilosavljevic]

@dotnet/product-construction @dotnet/source-build

[ViktorHofer]

Few quick questions:

1. Is this about VMR or sdk VMR pipelines?
2. How is this different from the jobs that run as part of PRs? Maybe I'm misunderstanding and you are proposing to add a pipeline that is identical to PRs but can be manually queued?

[NikolaMilosavljevic]

Few quick questions:

1. Is this about VMR or sdk VMR pipelines?
2. How is this different from the jobs that run as part of PRs? Maybe I'm misunderstanding and you are proposing to add a pipeline that is identical to PRs but can be manually queued?

1. This is about VMR.
2. Yes, essentially a new pipeline that would run similar/same jobs we run in PRs, but that can be manually queued.

Two goals:

1. Have a way to request a build that would run a job that builds tests
2. Have a way to validate a VMR change without running the main, heavy-weight pipeline

[ViktorHofer]

OK that makes sense and adds value. Could we re-use the existing dotnet-unified-build pipeline by making it possible to set the type as a queue trigger similar to the sign switch that can be specified? I think that would simplify the UX.

[NikolaMilosavljevic]

OK that makes sense and adds value. Could we re-use the existing dotnet-unified-build pipeline by making it possible to set the type as a queue trigger similar to the sign switch that can be specified? I think that would simplify the UX.

That seems interesting - we'd have Full and Verification values, for instance. I think Full would have to be the default, to ensure scheduled and triggered runs aren't affected.

[ViktorHofer]

Yes, we would keep the current defaults. We have the different scopes listed here: https://github.com/dotnet/sdk/blob/b1d1a0d44487a18a0989ced4fb4c23530da3744f/src/SourceBuild/content/eng/pipelines/ci.yml#L155-L164

@mmitche @akoeplinger are you ok with the proposed change to be able to set the scope when triggering the dotnet-unified-build pipeline?

[akoeplinger]

Sounds good to me

[mmitche]

I really like the idea of just making the queue time parameters a bit more descriptive Full and Verification. That also gives us the opportunity to add new flavors (e..g Tall Stacks vs. Short Stacks) and potentially check-boxes to limit the number of OS's built. So someone only interested in the Windows full stacks, could make that happen.

[NikolaMilosavljevic]

I presume we still want to maintain different set of jobs for different verification scenarios, i.e. internal vs public, CI vs PR.

I do intend to add one or two jobs for internal CI verification that would build tests. Currently we only do that in public CI or public PR.