Description
The prepare signed artifacts stage downloads the full set of artifacts (nupkgs, etc) that all jobs contribute to, does some prep work on them such as signing, then uploads this large set back up to the build so that Arcade stages can publish them.
The reupload has been hitting some Azure rate limits and making official builds fail very frequently. We got a temporary rate limit increase but need to get to a better place. ICM 166161231, Msft email thread "AzDO Artifact Pipeline"
We don't yet know exactly how the rate limit works (API rate vs. size rate vs. ???), but we know that migrating to Pipeline Artifacts is the way to go.
This means:
- Stop using logging commands to upload these artifacts, because it's not supported with Pipeline Artifacts anymore: Add logging command to upload pipeline artifacts. microsoft/azure-pipelines-tasks#11028
- This will cause the MSBuild code to be more tightly coupled with the yaml, but that isn't a huge concern for me with this specific stage/job.
- Upload all the artifacts in one go, probably with
PublishPipelineArtifact@1 - Still use logging commands to upload "tricky" files like the asset manifests. This is Arcade infrastructure that we should handle uniformly across the repos IMO. We could try moving the manifests to a staging directory, assembling a manifest of things to upload that we handle ourselves in yaml, or something like that, but let's leave that to another time. These are small and few.
Other places have more significant reasons to use logging commands, so they'll be harder to switch over. The prepare signed artifacts stage is low-hanging fruit. I think it'd take me a handful of days.
/cc @dotnet/runtime-infrastructure