mirrors."docker.io" is ignored #5612
Comments
|
Is your registry v2 API really at I suspect that what you want is: mirrors:
"docker.io":
endpoint":
- "http://10.43.251.176:15001/v2"
rewrite:
"(.*)": "docker.io/$1"
"ghcr.io":
endpoint:
- "http://10.43.251.176:15001/v2"
rewrite:
"(.*)": "ghcr.io/$1"
"quay.io":
endpoint:
- "http://10.43.251.176:15001/v2"
rewrite:
"(.*)": "quay.io/$1"
"k8s.gcr.io":
endpoint:
- "http://10.43.251.176:15001/v2"
rewrite:
"(.*)": "k8s.gcr.io/$1" |
|
I would also encourage you to try this again with a more recent release of RKE2. We have made significant changes to containerd registry mirror handling in the last few releases, and no longer write the mirror config into config.toml at all. |
|
@brandond We are in an environment where it is so difficult to upgrade rke2... 🥲 |
|
A better choice would be to use a namespace-aware registry mirror. containerd already passes the original registry namespace as the |
|
You're going to need to upgrade one way or another in order to get this fixed, regardless of whether it's been fixed already, or will be fixed in the future. |
|
I will say that there still does appear to be something wrong with handling of mirror endpoint paths that contain Can you modify your docker-path-proxy thing to use the ns query parameter, instead of embedding it in the registry API path? |
|
Since docker-path-proxy is behind nexus, it cannot be used for ns unless nexus is modified. I also checked that the problem still exists in the latest rke2. Just curious, if not in config.toml, where is the registry information stored in the latest rke2? Is containerd's config.toml completely deprecated? |
|
My understanding was that nexus already supports the ns query parameter for passing through the original registry namespace. If you're using nexus, why do you need to also modify the path? |
|
I understand that nexus does not have any processing for "ns" queries. docker-path-proxy behind nexus recognizes the first part of the path as a domain, so when nexus finds an image called "(proxy)/docker.io/hello/world:1.2.3", it goes to hello/world in docker.io. |
|
I'm not sure how, but I've worked with several users who have nexus doing the same thing with the ns query parameter that you're doing with the path. I'm not sure if they're doing it directly in nexus, or with nginx or haproxy in front of it rewriting the requests. Based on what you're doing, I suspect it may be the latter. |
|
OK this had me stumped for WAYYYYYY too long. There is no bug in RKE2. You have an extra trailing quote in your YAML. It's valid, but the key is mirrors:
"docker.io":
endpoint":
- "http://10.43.251.176:15001/v2/docker.io/"My eyes just didn't want to see it, I had to convert it to JSON to actually notice it: {
"mirrors": {
"quay.io": {
"endpoint": [
"http://10.43.251.176:15001/v2/quay.io/"
]
},
"k8s.gcr.io": {
"endpoint": [
"http://10.43.251.176:15001/v2/k8s.gcr.io/"
]
},
"docker.io": {
"endpoint\"": [
"http://10.43.251.176:15001/v2/docker.io/"
]
},
"ghcr.io": {
"endpoint": [
"http://10.43.251.176:15001/v2/ghcr.io/"
]
}
}
} |
|
@brandond OMG... I didn't see it at all. I'm sorry for my mistake, and thank you for finding my mistake. |
Environmental Info:
RKE2 Version:
Node(s) CPU architecture, OS, and Version:
Cluster Configuration:
2 servers
Describe the bug:
Only docker.io in registries.yaml is ignored.
Steps To Reproduce:
Install RKE2
Write
/etc/rancher/rke2/registries.yamlRestart
systemctl restart rke2-server.serviceWrong containerd config.toml
/var/lib/rancher/rke2/agent/etc/containerd/config.toml
Expected behavior:
Like everything else, docker.io also needs to have an endpoint applied.
Actual behavior:
Only docker.io in registries.yaml is ignored.
The text was updated successfully, but these errors were encountered: