emsdk and runtime workloads should publish workload blobs to the right places, with good paths

[mmitche]

In emsdk and (possibly) runtime, the manifest blobs are being dropped from publishing. See the "Workloads" artifact here: https://dev.azure.com/dnceng/internal/_build/results?buildId=1261503&view=artifacts&pathAsName=false&type=publishedArtifacts

This involves a couple things:

• Update ItemsToPushToBlobFeed to include those artifacts
• Set the RelativeBlobPath correctly for those artifacts. The form should be: emsdk|Runtime/<Full Nuget Version Number>/workloads/<file>

Note that the full nuget version number is the non-stable version numer. Since some of these artifacts stabilize their version numbers, it's highly important that build to build these paths be unique. It's also important for the hygiene of the storage location that no extra /'s or preceding /'s appear, and that we use the same top level prefixes that other repos do (component/full version/...)

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In emsdk and (possibly) runtime, the manifest blobs are being dropped from publishing. See the "Workloads" artifact here: https://dev.azure.com/dnceng/internal/_build/results?buildId=1261503&view=artifacts&pathAsName=false&type=publishedArtifacts

This involves a couple things:

• Update ItemsToPushToBlobFeed to include those artifacts
• Set the RelativeBlobPath correctly for those artifacts. The form should be: emsdk|Runtime/<Full Nuget Version Number>/workloads/<file>

Note that the full nuget version number is the non-stable version numer. Since some of these artifacts stabilize their version numbers, it's highly important that build to build these paths be unique. It's also important for the hygiene of the storage location that no extra /'s or preceding /'s appear, and that we use the same top level prefixes that other repos do (component/full version/...)

Author:	mmitche
Assignees:	-
Labels:	area-Infrastructure, untriaged
Milestone:	-

[directhex]

Just to confirm, @mmitche, the top level should be "emsdk" in your example, i.e. in the "dotnet-emsdk-official" job, there's a "emsdk" artifact push containing a directory "6.0.4.37702", and inside it there should be files?

And which files precisely are you expecting to see in there which currently you aren't? The .wixpack.zip files? Or something else?

[mmitche]

The build's artifacts don't matter at that much. What appears in PackageArtifacts and BlobArtifacts and the MergedManifest (inside BlobArtifacts) are what matters. These are automatically populated by whatever is in ItemsToPushToBlobFeed before the call to publish. These items tell the push what files need to be published for other builds (or customers) to consume, as well as their relative paths within blob storage.

See aspnetcore's logic for a good template. https://github.com/dotnet/aspnetcore/blob/main/eng/Publishing.props#L58-L85

[directhex]

@mmitche how's this https://maestro-prod.westus2.cloudapp.azure.com/1647/https:%2F%2Fgithub.com%2Fdotnet%2Femsdk/latest/graph

[mmitche]

I think that is going to be right for stable builds, but it still appears to be missing the workload artifacts.

[directhex]

And the missing artifacts from the BAR in this case are the .msi files from artifacts/VSSetup/Release/Insertion/? Add those and it's fine? Or all the files from that directory? They're all uploaded as AzDO artifacts, because the artifact upload task in AzDO is poorly written and doesn't understand wildcards properly, so there's no clean way to upload only some files out of a directory.

[mmitche]

@joeloff will know for sure, but I think it's all of them.

[joeloff]

The MSIs and the corresponding .json files. Some of the JSON files will have GUIDy names (those are for the packs). VS components will have more readable names. For example, from a previous EMSDK build, we'd want

0032ac322c79a6fae1abaad560c186d9.json
776d91824f437135c5e73b8650768748.json
792b7cb63ec0e49081d170db33d0609c.json
80be127b0a95e217f38c0d2beca6ec43.json
ff7305e9530353f9890b2d5f2b698cfa.json
Microsoft.Emscripten.2.0.23.Node.win-x64.6.0.0-preview.7.21377.2.msi
Microsoft.Emscripten.2.0.23.Python.win-x64.6.0.0-preview.7.21377.2.msi
Microsoft.Emscripten.2.0.23.Sdk.win-x64.6.0.0-preview.7.21377.2.msi
microsoft.net.sdk.emscripten.6.0.json
Microsoft.NET.Workload.Emscripten.Manifest-6.0.100.6.0.0-preview.7.21377.2-x64.msi
Microsoft.NET.Workload.Emscripten.Manifest-6.0.100.6.0.0-preview.7.21377.2-x86.msi