Description
Objective:
I'm trying to install k8s using kubespray with offline mode.
Observations:
I have been able to successfully bootstrap a cluster using k8s and offline mode however it wasn't without some issues and seeking guidance on how to implement some of the "hacks" I put in place.
Questions:
1. How do you properly set the username and password for "{{ file_repos }}" used to download the binaries?
What did I to workaround this?
Redefine this section https://github.com/kubernetes-sigs/kubespray/blob/release-2.21/roles/download/defaults/main.yml#L1985-L1996 in my inventory and set the values for username and password. But I feel like this may not be right way and seeking guidance on what the right way would be.
Suggestion
Update the docs for offline use that explain how to properly setup the authentication aspects on the file repo.
2. I have a similar situation with the containers downloaded for offline use. I have archived all the docker containers into a secure private registry on prem. The issue is it does not support unauthenticated/anonymous requests and apart from that depending on the container runtime you use, in my case default containerd, you also don't have the cli installed by default on your k8s cluster. How do you solve this chicken and egg problem? I don't see a configuration or variables to specify for nerdctl that would allow the playbook to login before it attempts to pull down the images from the secure registry. In https://github.com/kubernetes-sigs/kubespray/blob/release-2.21/roles/download/tasks/download_container.yml#L56 I also don't see any code that allows me to login via nerdctl. I also don't see it in https://github.com/kubernetes-sigs/kubespray/blob/release-2.21/roles/download/tasks/prep_download.yml which is a bit odd.
What did I to workaround this?
This one was particularly egregious to me but since I'm doing it in my dev environment I did it once to understand how this all works. After I figured out that I had to log into my private secure registry, and the kubepsray playbook failed. I ssh'ed into each of the nodes in my cluster(3) and did a manual nerdctl login and reran the kubespray playbook. While this works, this solution does not scale. And I also don't think it is a good idea for me to write mine own playbook to do a login after the fact.
Suggestion:
- I think what could be done for this situation is to possible split up this task https://github.com/kubernetes-sigs/kubespray/blob/release-2.21/roles/download/tasks/main.yml#L19 into two. One for binaries first and another for images third. Now in between the two tasks we could have an optional login task for your offline secure registry. This would also have to take into account the container runtime and binary you use. What I don't know is if nerdctl can use the
/etc/containerd.confconfig file for logging into the registry but in my case I tried to look into this initially and couldn't find anything.
Thank you for reading my giant wall of text, I hope that I was able to convey myself and look forward to your guidance response.
Thanks,
-MrMcMuffinz.