[Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=c1:c2) #211

fabianbrash · 2020-04-30T12:38:44Z

I removed the bash version and installed the go version on ubuntu and now I am receiving the error message "kubeconfig error: failed to load: cannot determine kubeconfig path: multiple files in KUBECONFIG are currently not supported"

ahmetb · 2020-04-30T15:19:54Z

Yeah this is a major point not implemented in Go version yet as I’m not fully sure how to do it in terms of modifying files when there are multiple of them (which one do you set current-context on).

I will pin this issue as others will most likely hit it too.

fabianbrash · 2020-04-30T17:03:50Z

so I've only been using k8s for about 4 months now so I'm still learning(probably never stop) but I'm still trying to wrap my head around the config files especially when you have more than 1, they can be merged in memory and that's what I've used I will just try and merge them all to a single file and export that to $KUBECONFIG, there is a --raw or --flatten option??

ahmetb · 2020-04-30T17:16:25Z

Yes, they can be merged in-memory or into a file. Read https://ahmet.im/blog/mastering-kubeconfig/. However, in this situation I need to study how to modify which file when user specifies KUBECONFIG=file1:file2:....

fabianbrash · 2020-04-30T17:46:49Z

thanks I'll have a look

fabianbrash · 2020-04-30T17:47:36Z

btw awesome work!!!

sedooe · 2020-05-02T01:46:51Z

Yes, they can be merged in-memory or into a file. Read https://ahmet.im/blog/mastering-kubeconfig/. However, in this situation I need to study how to modify which file when user specifies KUBECONFIG=file1:file2:....

I dug through kubectl and saw that it uses this to get the config file to write current context into.

Here I tried to add multiple kubeconfig support quickly, seems working but not sure if it's good enough for a PR or if you'd be interested with such a big one. Actually it looks bigger than it is because I had to move somethings around because of a cyclic dependency.

ahmetb · 2020-05-31T22:30:31Z

More context:

hodbn · 2020-11-20T12:42:19Z

@sedooe This is useful :) Are you planning to create a PR?

sedooe · 2020-11-20T12:54:42Z

@sedooe This is useful :) Are you planning to create a PR?

At that time we talked with @ahmetb and agreed on some improvements but unfortunately I couldn't find time to implement them since then. Feel free to use my fork as a baseline if you're planning to create a PR :)

retr0h · 2021-05-27T00:02:59Z

I use the kubecctx to manage multiple files. However, I'd rather see this in kubectx.

bdronneau · 2021-07-20T12:01:20Z

Hello,
any updates on this behavior ?

At that time we talked with @ahmetb and agreed on some improvements but unfortunately I couldn't find time to implement them since then. Feel free to use my fork as a baseline if you're planning to create a PR :)

Do you list somewhere the improvements needed to have a PR good enough ?

Best

willzhang · 2021-08-13T17:07:47Z

we need:

kuectx add kubeconfig-1,kueconfig-2,kueconfig-3 > /root/.kube/config

# cat  /root/.kube/config
apiVersion: v1
kind: Config
preferences: {}
clusters:
- cluster:
    server: https://xxxxx:6443
    certificate-authority-data: xxxxx
  name: cluster1 #第一个集群名称
- cluster:
    server: https://xxxxx:6443
    certificate-authority-data: xxxxx
  name: cluster2 #第二集群名称
contexts:
- context:
    cluster: cluster1 #第一个集群名称
    user: user1 #第一个集群用户
  name: cluster1 #第一个集群名称
- context:
    cluster: cluster2 #第二集群名称
    user: user2 #第二个集群用户
  name: cluster2 #第二集群名称
users:
- name: user1 #第一个集群context中的user
  user:
    client-certificate-data: xxxxxxxxxxx
    client-key-data: xxxxxxxxxxxxxxxxxxx
- name: user2 #第二个集群context中的user
  user:
    client-certificate-data: xxxxxxxxxxx
    client-key-data: xxxxxxxxxxxxxxxxxxx

ahmetb · 2021-08-13T17:16:04Z

we need:

kuectx add kubeconfig-1,kueconfig-2,kueconfig-3 > /root/.kube/config

read above @willzhang:

Yes, they can be merged in-memory or into a file. Read ahmet.im/blog/mastering-kubeconfig.

willzhang · 2021-08-17T16:23:39Z

thanks,it's useful , but user must different in each kubeconfig

KUBECONFIG=kubeconfig1.json:kubeconfig2.json kubectl config view --merge --flatten > config
cp config /root/.kube/config

[root@master ~]# kubectx
kubernetes-admin2@disaster-cluster
kubernetes-admin@prod-cluster

ahmetb · 2021-08-17T16:36:09Z

thanks,it's useful , but user must different in each kubeconfig

What you want is a tool to wrangle kubeconfig. Kubectx is only for switching "context"s.

https://krew.sigs.k8s.io/plugins/ has plenty of plugins like konfig, config-registry, config-cleanup that can help you.

tekumara · 2022-01-23T02:12:45Z

In the meantime, could the behaviour here just be the same as the bash version? ie: just set the context on the first file.

This would at least make the go version backwards compatible, and tricks like having a temp kubeconfig file would still work.

Happy to raise a PR to do so if it seems worthwhile.

requires switching to the bash version of kubectx because this is not yet supported by the golang version see ahmetb/kubectx#211

yuzhichang · 2022-05-22T08:29:06Z

If multiple kubeconfig are merged into one according to @willzhang , I can use kubectl config get-contexts to list contexts, kubectl config use-context to switch context.

I add following alias to ~/.bash_profile:

alias kcg="kubectl config get-contexts"
alias kcu="kubectl config use-context"

I wander whtat's the value of kubectx on contexts management?

cprivitere · 2022-05-23T13:56:13Z

Sure, for me, typing kctx and picking the context from a quick pop up menu is much easier than typing out the name of the context.
Additionally, getting everything merged into one file doesn't work well for everyone.

omniproc · 2022-08-26T09:42:42Z

Yeah this is a major point not implemented in Go version yet as I’m not fully sure how to do it in terms of modifying files when there are multiple of them (which one do you set current-context on).

I will pin this issue as others will most likely hit it too.

Best effort is good enough for most cases. We assume, altought there are multiple ctx across multiple files, every one is named unique. As long as there are no conflicting names it's not an problem.
So a possible strategy would be:

get all .kubeconfig files from ~/.kube
merge them into a single object kubectx works on
if there's no conflict switch to the selected kubectx
do so by coping the relevant objects to the .kube/config if they're not already there and enableing the ctx
if there's a conflict and the user didn't select that conflicting ctx: ignore it (just remove all conflicts from the single object generated on top)
if there is a conflict and the user did select the conflicting context don't do anything but fail with a error informing the user about the issue

I don't see a problem adding new, non conflicting data from other .kubeconfigs to the main config.

ic4-y · 2023-05-04T18:26:51Z

Since there appears to be not much happening here: I stumbled across this today. Very sad that the Go version doesn't implement this yet. The packaging of my distro (NixOS) no longer provides for the bash script version of kubectx. Love to see this getting added in the not so distant future.

cprivitere · 2023-05-04T18:37:32Z

For folks finding this who are wondering how to still use kubectx when you have lots of kubeconfig files, here's an example for your .zshrc file to generate a KUBECONFIG variable that kubectx can read.

#Lots of kubeconfig files!
# If there's already a kubeconfig file in ~/.kube/config it will import that too and all the contexts
DEFAULT_KUBECONFIG_FILE="$HOME/.kube/config"
if test -f "${DEFAULT_KUBECONFIG_FILE}"
then
  export KUBECONFIG="$DEFAULT_KUBECONFIG_FILE"
fi
# Your additional kubeconfig files should be inside ~/.kube/ and named kubeconfig.*.yml or kubeconfig.*.yaml
OIFS="$IFS"
IFS=$'\n'
for kubeconfigFile in `find "$HOME/.kube/" -type f -name "kubeconfig.*.yml" -o -name "kubeconfig.*.yaml"`
do
  export KUBECONFIG="$kubeconfigFile:$KUBECONFIG"
done
IFS="$OIFS"

I also find myself using kubie when I need to lock a particular session to a specific kubeconfig context/namespace. It's not as quick as kctx, but the session you set up with kubie will stay on that kubeconfig even if you use kctx to change it everywhere else.

ahmetb/kubectx#211

air3ijai · 2023-09-15T19:24:53Z

For folks finding this who are wondering how to still use kubectx when you have lots of kubeconfig files, here's an example for your .zshrc file to generate a KUBECONFIG variable that kubectx can read.

Your solution does not work for me or maybe I miss something?

macOS - zsh / kubectx_v0.9.5_darwin_arm64.tar.gz

KUBECONFIG=/Users/user/.kube//kubeconfig.cluster-1.yaml:/Users/user/.kube//cluster-2.yaml:

Linux - bash / kubectx_v0.9.5_linux_x86_64.tar.gz

KUBECONFIG=/root/.kube/kubeconfig.cluster-1.yaml:/root/.kube/kubeconfig.cluster-2.yaml:

Can you please provide an example of the KUBECONFIG variable with multiple files?

due to an issue with kubectx (ahmetb/kubectx#211) I need to switch to a single kubeconfig keeping multiple kubeconfig collection … will need it when a new config is added … in this case: - add new kubeconfig to ~/.kube - switch to the multi-config KUBECONFIG and apply - run `kubectl config view --flatten > ~/.kube/config` - change back to the single-config KUBECONFIG and apply

Pinimo · 2024-02-26T21:29:55Z

Easy workaround based on this StackOverflow answer. Write your different configs in multiple files and merge them whenever you need, for instance with the following in your ~/.zshrc (or other shell startup script):

alias kconfig='KUBECONFIG=~/.kube/config.microk8s:~/.kube/config.other kubectl config view --flatten > ~/.kube/config'

This will export a merged configuration to ~/.kube/config live. Note that it will also override the previous configuration, however. Having an alias is useful in my case (one file is dynamic), however it is also possible to use directly this:

KUBECONFIG=~/.kube/config.microk8s:~/.kube/config.other kubectl config view --flatten > ~/.kube/config

den-is · 2024-05-28T15:52:18Z

year 2024, that's sad :(

I've been using kubectx like forever.

But used it in a single cluster environments.
Or environments with combined kubeconfig. (old approach, not using it for a long time)
And actually was using kubectx in multi-x100 cluster environments on mac!

And today I was surprised to discover that brew brew-installed kubectx is actually a bash flavor, when my go-flavor install on linux has failed with multiple kubeconfig files.

ahmetb changed the title ~~kubeconfig error~~ [Go] does not work with multiple kubeconfig files Apr 30, 2020

ahmetb added the Go implementation label Apr 30, 2020

ahmetb pinned this issue Apr 30, 2020

ahmetb changed the title ~~[Go] does not work with multiple kubeconfig files~~ [Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=a:b:c) Aug 17, 2021

ahmetb changed the title ~~[Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=a:b:c)~~ [Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=c1:c2) Aug 17, 2021

tekumara added a commit to tekumara/setup-ubuntu that referenced this issue Jan 26, 2022

feat: support multiple kube config files

0a76fd2

requires switching to the bash version of kubectx because this is not yet supported by the golang version see ahmetb/kubectx#211

z0al added a commit to z0al/dotfiles that referenced this issue Jul 31, 2023

chore(kubectx): pin v0.8.0 for multi-config support

6941775

ahmetb/kubectx#211

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=c1:c2) #211

[Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=c1:c2) #211

fabianbrash commented Apr 30, 2020

ahmetb commented Apr 30, 2020

fabianbrash commented Apr 30, 2020

ahmetb commented Apr 30, 2020

fabianbrash commented Apr 30, 2020

fabianbrash commented Apr 30, 2020

sedooe commented May 2, 2020 •

edited

Loading

ahmetb commented May 31, 2020

hodbn commented Nov 20, 2020

sedooe commented Nov 20, 2020

retr0h commented May 27, 2021 •

edited

Loading

bdronneau commented Jul 20, 2021

willzhang commented Aug 13, 2021

ahmetb commented Aug 13, 2021

willzhang commented Aug 17, 2021 •

edited

Loading

ahmetb commented Aug 17, 2021

tekumara commented Jan 23, 2022

yuzhichang commented May 22, 2022 •

edited

Loading

cprivitere commented May 23, 2022 •

edited

Loading

omniproc commented Aug 26, 2022

ic4-y commented May 4, 2023

cprivitere commented May 4, 2023 •

edited

Loading

air3ijai commented Sep 15, 2023

Pinimo commented Feb 26, 2024

den-is commented May 28, 2024

[Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=c1:c2) #211

[Go] does not work with multiple kubeconfig files (e.g. KUBECONFIG=c1:c2) #211

Comments

fabianbrash commented Apr 30, 2020

ahmetb commented Apr 30, 2020

fabianbrash commented Apr 30, 2020

ahmetb commented Apr 30, 2020

fabianbrash commented Apr 30, 2020

fabianbrash commented Apr 30, 2020

sedooe commented May 2, 2020 • edited Loading

ahmetb commented May 31, 2020

hodbn commented Nov 20, 2020

sedooe commented Nov 20, 2020

retr0h commented May 27, 2021 • edited Loading

bdronneau commented Jul 20, 2021

willzhang commented Aug 13, 2021

ahmetb commented Aug 13, 2021

willzhang commented Aug 17, 2021 • edited Loading

ahmetb commented Aug 17, 2021

tekumara commented Jan 23, 2022

yuzhichang commented May 22, 2022 • edited Loading

cprivitere commented May 23, 2022 • edited Loading

omniproc commented Aug 26, 2022

ic4-y commented May 4, 2023

cprivitere commented May 4, 2023 • edited Loading

air3ijai commented Sep 15, 2023

Pinimo commented Feb 26, 2024

den-is commented May 28, 2024

sedooe commented May 2, 2020 •

edited

Loading

retr0h commented May 27, 2021 •

edited

Loading

willzhang commented Aug 17, 2021 •

edited

Loading

yuzhichang commented May 22, 2022 •

edited

Loading

cprivitere commented May 23, 2022 •

edited

Loading

cprivitere commented May 4, 2023 •

edited

Loading