Duplicate ref packs are getting uploaded

[agocke]

See https://dev.azure.com/dnceng/internal/_build/results?buildId=2459834&view=logs&j=0412dcf5-478c-51bc-8fd3-1e1653027e8b&t=8fbfd639-c99a-52bf-214f-b6ce4811bf9d

Seems unusual that it's uploading ref packs. I don't see why it should be creating or uploading ref packs.

[agocke]

@akoeplinger Any ideas?

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

This doesn't look bionic or native AOT specific. This particular leg failed, but I don't see an indication it's because of the duplicate:

Fail to upload '/mnt/vss/_work/1/a/IntermediateArtifacts/NativeAOTRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.5.24274.5.symbols.nupkg' due to 'Bad Request'.
Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: Bad Request
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Byte[] contentId, Int64 fileLength, Boolean isGzipped, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Int32 chunkRetryTimes, Boolean uploadFirstChunk, Object userState)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Boolean uploadFirstChunk, Object userState, Boolean compressStream)
   at Microsoft.VisualStudio.Services.Agent.Worker.Build.FileContainerServer.UploadAsync(IAsyncCommandContext context, Int32 uploaderId, CancellationToken token) in /mnt/vss/_work/1/s/src/Agent.Worker/Build/FileContainerServer.cs:line 253

Clicking different legs in the same run, I see e.g.:

Uploading 'IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.5.24274.5.nupkg' (50%)

in the Mono LLVM AOT leg. I wonder if there's downstream logic somewhere to only publish one in the NuGet feed and throw away the rest.

[akoeplinger]

This particular leg failed, but I don't see an indication it's because of the duplicate

The Bad Request is that indication, AzDO throws that when we try to upload duplicates at the same time. It's racy, so we'll only hit that when two legs are uploading at the same time (my understanding is we overwrite otherwise).

Seems unusual that it's uploading ref packs. I don't see why it should be creating or uploading ref packs.

Yes. I think we have that problem in other legs too, it just happens very infrequently.