Add Flat flow preview flow script

[dkurepa]

To double check:

dotnet/arcade-services#4533

• The right tests are in and the right validation has happened. Guidance: https://github.com/dotnet/arcade/blob/main/Documentation/Validation.md

[dkurepa]

I don't think this PR is good to merge yet, it's missing some information, mostly about which branches and channels tooling repos should use, and also branches that currently have main as their default branch currently. I'm guessing we'll have to start branching these repos too, and then use those new branches.

@ViktorHofer, @mmitche could you provide me with this info?

[mmitche]

@dkurepa brings up an interesting question about what to do for repos that do not branch for a preview. There are many of these. Do we add backflow? We can't really add backflow because

flowchart LR
  roslyn["roslyn main"]
  VMRmain["VMR main"]
  VMRp5["VMR preview 5"]
  
  roslyn--FF-->VMRmain
  roslyn--FF-->VMRp5
  VMRmain--BF-->roslyn
  VMRp5 -. BF?? .-> roslyn

Loading

My sense is that repos that do not branch should not get backflow. Repos may choose not to branch if

• They do not take "interesting" dependencies on backflow (e.g. runtime). Backflow exists to ensure that repos are validated against what they built against in the VMR. This takes the place of package dependency flow in our old preview branching structure. E.g. prior to the VMR, runtime flows to aspnetcore. which both causes aspnetcore to validate against the runtime it will ship against, as well as build its own shipping assets against that runtime. Roslyn, however, does not have live flow from runtime. Therefore no packages should need to flow on backflow.
• We do not make changes in the VMR to those repos - This basically just says that the sources should match. This is again related to validation. You want to have validated in the roslyn repo all the source you will ship. Backflow supplies no additional validation benefit.

This pretty much directly maps to what happens in normal dependency flow today. roslyn can flow to multiple output SDK branches, but couldn't take updates from multiple SDK branches.

I think we will need validation of the second point. I think in an ideal world, you could set up backflow from two branches to one branch. Flow would normally go from vmr main -> roslyn main. But, if someone makes a change in VMR p5 and it flows to roslyn main, we need to examine the change and decide what the state of the world needs to be.

/cc @premun

[mmitche]

No need to merge this immediately. Just a review. I'll use it to create the preview flow tomorrow and then we can merge after a day or so once we see it's working as expected.

[mmitche]

May need an excludeAssets * on templating, depending on the outcome of dotnet/templating#9060

[ViktorHofer]

May need an excludeAssets * on templating, depending on the outcome of dotnet/templating#9060

Can we copy the excludeAssets metadata from the corresponding main subscriptions to avoid another source of truth?

[premun]

@mmitche FYI I just found out all the tooling repos have their forward flow subscriptions created without the TargetDirectory property somehow.
These subscriptions are now disabled so maybe it happened during their disabling but also might be a bug in the scripts here.

[dkurepa]

May need an excludeAssets * on templating, depending on the outcome of dotnet/templating#9060

Can we copy the excludeAssets metadata from the corresponding main subscriptions to avoid another source of truth?

I just went through all of the main subs and added the excluded assets to the script

[ViktorHofer]

What I meant earlier was whether we can fetch the list of excluded assets from the main subscription so that we don't need to hardcode them here. Otherwise we would need to keep this list up-to-date.

[dkurepa]

ahh, right, good idea, will add that instead

[ViktorHofer]

Now I wonder if we would want a a darc copy-subscription --id X command that would allow overriding parameters.