Skip to content

Make Helm Plugins available to all users #62

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Angelin01
Copy link

When running the image as a non-root user, the helm plugins can't be found because they are normally located at the user's home folder. During installation, they are installed to /root, which is not globally readable:

> docker run --rm -ti -u "1000:1000" alpine/k8s:1.27.4 helm unittest
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.

But if we configure the HELM_PLUGINS environment variable, Helm will install AND look for the plugins at that folder. By making the folder globally readable, we allow any user to use the plugins (without being able to modify them):

> docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:local helm unittest tests/dummy-chart

### Chart [ dummy-chart ] tests/dummy-chart

 PASS  images test      tests/dummy-chart/tests/images_test.yaml
 PASS  labels test      tests/dummy-chart/tests/labels_test.yaml
 PASS  names test       tests/dummy-chart/tests/names_test.yaml

Charts:      1 passed, 1 total
Test Suites: 3 passed, 3 total
Tests:       27 passed, 27 total
Snapshot:    0 passed, 0 total
Time:        38.429632ms

This is a much simpler change than #55 but should already help with people using the image in CI/CD environments where one can't use the root user.

@Angelin01
Copy link
Author

@ozbillwang Sorry, I noticed you gave it a 👍 but I don't know if you need anything else to merge this. If you do, feel free to share here and I'll get on it right away.

@ozbillwang
Copy link
Collaborator

@ozbillwang Sorry, I noticed you gave it a 👍 but I don't know if you need anything else to merge this. If you do, feel free to share here and I'll get on it right away.

Thank you for the PR; it shows promise.

I'll need some time to verify the changes, so please be patient.

@ozbillwang
Copy link
Collaborator

ozbillwang commented Aug 3, 2023

So the default path is /root/.cache/helm/plugins currently.

second, I didn't see the different with the Dockerfile you changed

# with exist image
$  docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:1.27.4 helm unittest .
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.

# with local image built from your Dockerfile
$ docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:local helm unittest .
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.

# and it breaks the exist usage
$  docker run --rm -ti -v "$PWD:/apps" alpine/k8s:local helm unittest
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.

@Angelin01
Copy link
Author

Angelin01 commented Aug 3, 2023

That is strange. Did you build the image with a newer version of Helm? I've noticed that the default version on the Dockerfile is too old and does not support the environment variable:

angelo@irgeshud:~/tmp > git clone git@github.com:Angelin01/k8s.git -b patch-1
Cloning into 'k8s'...
remote: Enumerating objects: 412, done.
remote: Counting objects: 100% (239/239), done.
remote: Compressing objects: 100% (83/83), done.
remote: Total 412 (delta 168), reused 186 (delta 146), pack-reused 173
Receiving objects: 100% (412/412), 74.87 KiB | 324.00 KiB/s, done.
Resolving deltas: 100% (235/235), done.
angelo@irgeshud:~/tmp > cd k8s/
# NOTICE THE BUILD ARG HERE
angelo@irgeshud:~/tmp/k8s (patch-1|u=) > docker build . -t alpine/k8s:2023-08-03 --build-arg HELM_VERSION=3.12.2 --build-arg KUBECTL_VERSION=1.27.4
[+] Building 80.9s (20/20) FINISHED                                                                                                                                                                docker:default
 => [internal] load .dockerignore                                                                                                                                                                            0.0s
 => => transferring context: 2B                                                                                                                                                                              0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                         0.0s
 => => transferring dockerfile: 3.72kB                                                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                                                                                                             1.6s
 => [ 1/16] FROM docker.io/library/alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1                                                                                            0.0s
 => CACHED [ 2/16] RUN case `uname -m` in     x86_64) ARCH=amd64; ;;     armv7l) ARCH=arm; ;;     aarch64) ARCH=arm64; ;;     ppc64le) ARCH=ppc64le; ;;     s390x) ARCH=s390x; ;;     *) echo "un-supported  0.0s
 => CACHED [ 3/16] RUN . /envfile && echo $ARCH &&     apk add --update --no-cache curl ca-certificates bash git &&     curl -sL https://get.helm.sh/helm-v3.12.2-linux-${ARCH}.tar.gz | tar -xvz &&     mv  0.0s
 => [ 4/16] RUN mkdir -p "/helm-plugins" &&     chmod 755 "/helm-plugins"                                                                                                                                    0.4s
 => [ 5/16] RUN helm plugin install https://github.com/databus23/helm-diff && rm -rf /tmp/helm-*                                                                                                             6.4s
 => [ 6/16] RUN helm plugin install https://github.com/helm-unittest/helm-unittest && rm -rf /tmp/helm-*                                                                                                     5.6s
 => [ 7/16] RUN helm plugin install https://github.com/chartmuseum/helm-push &&     rm -rf /tmp/helm-*     /root/.local/share/helm/plugins/helm-push/testdata     /root/.cache/helm/plugins/https-github.co  4.5s 
 => [ 8/16] RUN . /envfile && echo $ARCH &&     curl -sLO https://storage.googleapis.com/kubernetes-release/release/v1.27.4/bin/linux/${ARCH}/kubectl &&     mv kubectl /usr/bin/kubectl &&     chmod +x /u  3.3s 
 => [ 9/16] RUN . /envfile && echo $ARCH &&     curl -sLO https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv3.8.1/kustomize_v3.8.1_linux_${ARCH}.tar.gz &&     tar xvzf kustomize  2.1s 
 => [10/16] RUN . /envfile && echo $ARCH &&     curl -sL "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_${ARCH}.tar.gz" | tar xz -C /tmp &&     mv /tmp/eksctl /usr/bin   3.2s 
 => [11/16] RUN apk add --update --no-cache python3 &&     python3 -m ensurepip &&     pip3 install --upgrade pip &&     pip3 install awscli &&     pip3 cache purge                                        21.3s 
 => [12/16] RUN apk add --update --no-cache jq yq                                                                                                                                                            3.7s 
 => [13/16] RUN . /envfile && echo $ARCH &&     authenticator=$(curl -fs https://api.github.com/repos/kubernetes-sigs/aws-iam-authenticator/releases/latest | jq --raw-output '.name' | sed 's/^v//') &&    10.8s 
 => [14/16] RUN apk add --update --no-cache gettext                                                                                                                                                          2.3s 
 => [15/16] RUN . /envfile && echo $ARCH &&     curl -L https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.18.1/kubeseal-0.18.1-linux-${ARCH}.tar.gz -o - | tar xz -C /usr/bin/ &&     chm  9.2s 
 => [16/16] WORKDIR /apps                                                                                                                                                                                    1.5s 
 => exporting to image                                                                                                                                                                                       4.5s 
 => => exporting layers                                                                                                                                                                                      4.5s 
 => => writing image sha256:37bf687ed8fe0a760dae05424e5fc84e418f088ee802e329706abb0f7a3e1da0                                                                                                                 0.0s 
 => => naming to docker.io/alpine/k8s:2023-08-03                                                                                                                                                             0.0s 
angelo@irgeshud:~/tmp/k8s (patch-1|u=) > docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:2023-08-03 helm unittest .

### Error:  Chart.yaml file is missing


Charts:      1 failed, 1 errored, 0 passed, 1 total
Test Suites: 0 passed, 0 total
Tests:       0 passed, 0 total
Snapshot:    0 passed, 0 total
Time:        3.060674ms

Error: plugin "unittest" exited with error

Maybe we should consider updating the default versions on the ARGs?

@Angelin01
Copy link
Author

Also, as a separate suggestion, maybe we can introduce some smoke tests during image build to help with these base verifications. Something like this:

RUN helm plugin install https://github.com/helm-unittest/helm-unittest && \
    rm -rf /tmp/helm-* && \
    helm unittest --help # This command will fail if the plugin is not installed properly and break the docker build.

I can probably whip this up in another PR.

@ozbillwang
Copy link
Collaborator

ozbillwang commented Aug 3, 2023

let me check, seems I build with old version of helm. The latest is:

helm version is 3.12.2

The Dockerfile need be reviewed, because several part has hardcoding the path of helm plugin and try to clean it. We need adjust them as well.

https://github.com/alpine-docker/k8s/blob/master/Dockerfile#L40-L42

@Angelin01
Copy link
Author

Angelin01 commented Aug 3, 2023

I've adjusted the removal of helm-pushs test data. The other files, like /root/.cache are controlled by other environment variables and don't need to be changed here. Here's the variables/paths that helm considers as a non root user:

/apps $ helm env
HELM_BIN="helm"
HELM_BURST_LIMIT="100"
HELM_CACHE_HOME="/.cache/helm"
HELM_CONFIG_HOME="/.config/helm"
HELM_DATA_HOME="/.local/share/helm"
HELM_DEBUG="false"
HELM_KUBEAPISERVER=""
HELM_KUBEASGROUPS=""
HELM_KUBEASUSER=""
HELM_KUBECAFILE=""
HELM_KUBECONTEXT=""
HELM_KUBEINSECURE_SKIP_TLS_VERIFY="false"
HELM_KUBETLS_SERVER_NAME=""
HELM_KUBETOKEN=""
HELM_MAX_HISTORY="10"
HELM_NAMESPACE="default"
HELM_PLUGINS="/helm-plugins"
HELM_REGISTRY_CONFIG="/.config/helm/registry/config.json"
HELM_REPOSITORY_CACHE="/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/.config/helm/repositories.yaml"

and as the root user:

/apps # helm env
HELM_BIN="helm"
HELM_BURST_LIMIT="100"
HELM_CACHE_HOME="/root/.cache/helm"
HELM_CONFIG_HOME="/root/.config/helm"
HELM_DATA_HOME="/root/.local/share/helm"
HELM_DEBUG="false"
HELM_KUBEAPISERVER=""
HELM_KUBEASGROUPS=""
HELM_KUBEASUSER=""
HELM_KUBECAFILE=""
HELM_KUBECONTEXT=""
HELM_KUBEINSECURE_SKIP_TLS_VERIFY="false"
HELM_KUBETLS_SERVER_NAME=""
HELM_KUBETOKEN=""
HELM_MAX_HISTORY="10"
HELM_NAMESPACE="default"
HELM_PLUGINS="/helm-plugins"
HELM_REGISTRY_CONFIG="/root/.config/helm/registry/config.json"
HELM_REPOSITORY_CACHE="/root/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/root/.config/helm/repositories.yaml"

@Angelin01
Copy link
Author

Angelin01 commented Aug 8, 2023

Hi, it's been a few days, I believe I've adjusted what needs adjusting, unless you want me to update the default versions for the ARGs or similar in this PR too. Anything else you need to merge this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants