Skip to content

Containers: fix handling of ContainerBaseImage without a registry. #32876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 30, 2023

Conversation

tmds
Copy link
Member

@tmds tmds commented May 28, 2023

These images are meant to be pulled from Docker Hub.

The wrong hostname was used for the registry ('registry-1.docker.io' vs 'docker.io').

The parsing of names like 'ubuntu/dotnet-runtime' lead to 'ubuntu' being treated as the registry.

This also enables using 'docker.io' as an alias to the Docker registry.

And adds the required 'library/' prefixing for short-names (like 'ubuntu' to 'library/ubuntu').

@baronfel ptal.

Fixes dotnet/sdk-container-builds#445
Fixes dotnet/sdk-container-builds#444
Fixes dotnet/sdk-container-builds#443

These images are meant to be pulled from Docker Hub.

The wrong hostname was used for the registry ('registry-1.docker.io' vs 'docker.io').

The parsing of names like 'ubuntu/dotnet-runtime' lead to 'ubuntu' being treated as the
registry.

This also enables using 'docker.io' as an alias to the Docker registry.

And adds the required 'library/' prefixing for short-names (like 'ubuntu' to 'library/ubuntu').
@tmds tmds requested a review from a team as a code owner May 28, 2023 06:59
@ghost
Copy link

ghost commented May 28, 2023

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost added the untriaged Request triage from a team member label May 28, 2023
@tmds tmds force-pushed the docker_base_image branch from 808bd67 to c629b92 Compare May 28, 2023 08:34
@tmds
Copy link
Member Author

tmds commented May 30, 2023

I've added a commit that addresses dotnet/sdk-container-builds#443 too.

@tmds tmds force-pushed the docker_base_image branch from 2644482 to 03d7aae Compare May 30, 2023 14:27
Copy link
Member

@baronfel baronfel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Lovely usability changes all around.

@baronfel
Copy link
Member

baronfel commented May 30, 2023

I'm testing this locally on baronfel/sdk-container-demo and running into a runtime crash when trying to target the ubuntu:latest base image:

C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(114,5): warning CONTAINER2020: FullyQualifiedBaseImageName does not specify a registry and will be pulled from Docker Hub. Please prefix the name with the image registry, for example: 'docker.io/<image>'. [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018: The "CreateNewImage" task failed unexpectedly. [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018: System.TypeInitializationException: The type initializer for 'System.Text.RegularExpressions.Generated.<RegexGenerator_g>F74B1AE921BCEFE4BA601AA541C7A23B1CA9711EA81E8FE504B5B6446748E035A__BearerParameterSplitter_0' threw an exception. [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:  ---> System.TypeInitializationException: The type initializer for 'System.Text.RegularExpressions.Generated.<RegexGenerator_g>F74B1AE921BCEFE4BA601AA541C7A23B1CA9711EA81E8FE504B5B6446748E035A__Utilities' threw an exception. [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:  ---> System.TypeLoadException: Could not load type 'System.Buffers.SearchValues' from assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at System.Text.RegularExpressions.Generated.<RegexGenerator_g>F74B1AE921BCEFE4BA601AA541C7A23B1CA9711EA81E8FE504B5B6446748E035A__Utilities..cctor() [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    --- End of inner exception stack trace --- [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at System.Text.RegularExpressions.Generated.<RegexGenerator_g>F74B1AE921BCEFE4BA601AA541C7A23B1CA9711EA8
1E8FE504B5B6446748E035A__BearerParameterSplitter_0..ctor() in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\System.Text.RegularExpressions.Generator\System.Text.RegularExpressions.Generator.RegexGenerator\RegexGenerator.g.cs:line 54 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at System.Text.RegularExpressions.Generated.<RegexGenerator_g>F74B1AE921BCEFE4BA601AA541C7A23B1CA9711EA8
1E8FE504B5B6446748E035A__BearerParameterSplitter_0..cctor() in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\System.Text.RegularExpressions.Generator\System.Text.RegularExpressions.Generator.RegexGenerator\RegexGenerator.g.cs:line 47 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    --- End of inner exception stack trace --- [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.NET.Build.Containers.AuthHandshakeMessageHandler.BearerParameterSplitter() in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\System.Text.RegularExpressions.Generator\System.Text.RegularExpressions.Generator.RegexGenerator\RegexGenerator.g.cs:line 26 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.NET.Build.Containers.AuthHandshakeMessageHandler.TryParseAuthenticationInfo(HttpResponseMessage msg, String& scheme, AuthInfo& authInfo) in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\AuthHandshakeMessageHandler.cs:line 50 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.NET.Build.Containers.AuthHandshakeMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\AuthHandshakeMessageHandler.cs:line 180 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.NET.Build.Containers.Registry.GetManifestAsync(String repositoryName, String reference, CancellationToken cancellationToken) in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\Registry.cs:line 219 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.NET.Build.Containers.Registry.GetImageManifestAsync(String repositoryName, String reference, String runtimeIdentifier, String runtimeIdentifierGraphPath, CancellationToken cancellationToken) in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\Registry.cs:line 147 [D:\Code\sdkcontainer-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.NET.Build.Containers.Tasks.CreateNewImage.ExecuteAsync(CancellationToken cancellationToken) in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\Tasks\CreateNewImage.cs:line 49 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.NET.Build.Containers.Tasks.CreateNewImage.Execute() in D:\Code\dotnet-sdk\src\Containers\Microsoft.NET.Build.Containers\Tasks\CreateNewImage.cs:line 32 [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [D:\Code\sdk-container-demo\sdk-container-demo.csproj]
C:\Users\chusk\.nuget\packages\microsoft.net.build.containers\8.0.100-dev\build\Microsoft.NET.Build.Containers.targets(196,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [D:\Code\sdk-container-demo\sdk-container-demo.csproj]

My first guess - I'm running that repo on the public preview4 bits - did something change in the Regex generator and since the SDK might be using a more recent runtime there's now a mismatch? I'm trying to validate the end-to-end user experience before I merge here.

Looks like a similar issue to dotnet/runtime#85913 (comment)?

@baronfel
Copy link
Member

Yes, I believe the issue is on my side. Using the SDK's eng/dogfood scripts I can get past this^

However, working against Docker's registry exposed another gap - docker seems to prefer/send OCI Image manifests, not 'docker' content types.

Here's what we send

application/json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.container.image.v1+json

However, those all return a 404. When I request application/vnd.oci.image.index.v1+json instead, however, Docker Hub returns a response. I fear that this means that some set of Docker Hub images are just unusable with our tooling at the moment, at least as base images. I'll log this as a separate issue that we should fix, but it doesn't block this usability enhancement (it just makes it more apparent that we need to fix).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
untriaged Request triage from a team member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DefaultRegistry does not work Cannot pull base image from docker.io Emit a warning when given a ContainerBaseImage that doesn't specify a registry
2 participants