Kaniko pod using default service account even whe build.cluster.serviceAccount is specified

[donovanrost]

Expected behavior

I would expect the Kaniko Pod to be deployed in the platform-gitlab-runners namespace with the service account gitlab-runner as specifed.

Actual behavior

Kaniko fails to deploy as the default service account is being used

Information

• Skaffold version: v2.10.0
• Operating system: Running in AlpineLinux on AmazonLinux2
• Installed via: skaffold.dev
• Contents of skaffold.yaml:

apiVersion: skaffold/v4beta8
kind: Config
metadata:
  name: config-service
build:
  cluster:
    namespace: platform-gitlab-runners
    serviceAccount: gitlab-runner
  artifacts:
    - image: config-service
      kaniko: {}
      sync:
        infer:
          - config_service/**/*
          - helm/**/*
  tagPolicy:
    gitCommit: {}

The skaffold.yaml continues with a handful of profiles which I can include if needed relevant.

Steps to reproduce the behavior

$ skaffold build -v debug --tag ${CI_COMMIT_SHA} --default-repo ${CI_REGISTRY_IMAGE} --file-output build-${CI_COMMIT_SHA}.json
time="2024-01-10T20:52:01Z" level=debug msg="skaffold API not starting as it's not requested" subtask=-1 task=DevLoop
time="2024-01-10T20:52:01Z" level=info msg="Skaffold &{Version:v2.10.0 ConfigVersion:skaffold/v4beta9 GitVersion: GitCommit:cbc665bfc1fe7253df466e70dd48e3851d935a3e BuildDate:2024-01-09T08:52:57Z GoVersion:go1.21.0 Compiler:gc Platform:linux/amd64 User:}" subtask=-1 task=DevLoop
time="2024-01-10T20:52:01Z" level=info msg="Loaded Skaffold defaults from "/root/.skaffold/config"" subtask=-1 task=DevLoop
time="2024-01-10T20:52:01Z" level=debug msg="config version out of date: upgrading to latest "skaffold/v4beta9"" subtask=-1 task=DevLoop
time="2024-01-10T20:52:01Z" level=debug msg="parsed 1 configs from configuration file /builds/ground-segment/spaceops/config-service/skaffold.yaml" subtask=-1 task=DevLoop
time="2024-01-10T20:52:01Z" level=info msg="map entry found when executing locate for &{config-service . 0xc0000d0070 { 0xc0008d5d40 } [] {[] []} [] } of type *latest.Artifact and pointer: 824643614400" subtask=-1 task=DevLoop
time="2024-01-10T20:52:01Z" level=info msg="Using kubectl context: " subtask=-1 task=DevLoop
time="2024-01-10T20:52:01Z" level=debug msg="getting client config for kubeContext: " subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=debug msg="no kube-context set and no kubeConfig found, attempting in-cluster config" subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=debug msg="Running command: [minikube version --output=json]" subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=debug msg="setting Docker user agent to skaffold-v2.10.0" subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=info msg="DOCKER_HOST env is not set, using the host from docker context." subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=debug msg="Running command: [docker context inspect --format {{.Endpoints.docker.Host}}]" subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=warning msg="Could not get docker context: starting command docker context inspect --format {{.Endpoints.docker.Host}}: exec: \"docker\": executable file not found in $PATH, falling back to the default docker host" subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=info msg="no kpt renderer or deployer found, skipping hydrated-dir creation" subtask=-1 task=DevLoop time="2024-01-10T20:52:01Z" level=debug msg="Running command: [kubectl config view --minify -o jsonpath='{..namespace}']" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="Running command: [helm version --client]" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="config version out of date: upgrading to latest \"skaffold/v4beta9\"" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="config version out of date: upgrading to latest \"skaffold/v4beta9\"" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="Command output: [version.BuildInfo{Version:\"v3.13.2\", GitCommit:\"\", GitTreeState:\"\", GoVersion:\"go1.21.3\"}\n], stderr: WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /builds/ground-segment/spaceops/config-service.tmp/KUBECONFIG\nWARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /builds/ground-segment/spaceops/config-service.tmp/KUBECONFIG\n" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="CLI platforms provided: \"\"" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="platform detection from active kubernetes cluster is not enabled" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="platforms selected for artifact \"config-service\": \"\"" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=debug msg="Using builder: cluster" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=info msg="build concurrency first set to 0 parsed from *runner.pipelineBuilderWithHooks[0]" subtask=-1 task=DevLoop time="2024-01-10T20:52:02Z" level=info msg="final build concurrency value is 0" subtask=-1 task=DevLoop Generating tags... config-service -> registry.<company>.team/ground-segment/spaceops/config-service/config-service:4c43e327f8e9c0b76fb6edb4ca9cf32b01081c0a time="2024-01-10T20:52:02Z" level=info msg="Tags generated in 99.885µs" subtask=-1 task=Build Checking cache... time="2024-01-10T20:52:03Z" level=debug msg="Found dependencies for dockerfile: [{pyproject.toml /code true 18 18} {poetry.lock /code true 18 18} {config_service /code/config_service false 21 21} {helm /code/helm false 22 22} {db-migrations /code/db-migrations false 23 23} {tests /code/tests false 24 24} {.env.test /code true 25 25} {alembic.ini /code true 26 26} {mypy.ini /code true 26 26} {README.md /code true 26 26} {logging-config.yaml /code true 27 27}]" subtask=-1 task=DevLoop time="2024-01-10T20:52:03Z" level=debug msg="Image config-service is remote because it has GoogleCloudBuild or pipeline.Build.Cluster" subtask=-1 task=DevLoop config-service: Not found. Building time="2024-01-10T20:52:03Z" level=info msg="Cache check completed in 1.034 second" subtask=-1 task=Build Starting build... Building [config-service]... time="2024-01-10T20:52:03Z" level=debug msg="getting client config for kubeContext: " subtask=-1 task=DevLoop
time="2024-01-10T20:52:03Z" level=debug msg="no kube-context set and no kubeConfig found, attempting in-cluster config" subtask=-1 task=DevLoop
time="2024-01-10T20:52:03Z" level=debug msg="Running command: [tput colors]" subtask=-1 task=DevLoop
time="2024-01-10T20:52:03Z" level=debug msg="error checking for color support: checking terminal colors: starting command tput colors: exec: "tput": executable file not found in $PATH" subtask=-1 task=DevLoop
***creating kaniko pod: pods is forbidden: User "system:serviceaccount:platform-gitlab-runners:default" cannot create resource "pods" in API group "" in the namespace "platform-gitlab-runners" ***
time="2024-01-10T20:52:03Z" level=debug msg="exporting metrics disabled" subtask=-1 task=DevLoop

[donovanrost]

Additionally, I'm on EKS using version 1.25.

[uluzox]

I can second this issue.

apiVersion: skaffold/v4beta5
kind: Config
metadata:
  name: test-project
build:
  artifacts:
    - image: image1
      kaniko: {}
  cluster:
    namespace: gitlab-runner
    serviceAccount: gitlab-runners

Generating tags...

• image1 -> image1:ec2453c
  Checking cache...
• image1: Not found. Building
  Starting build...
  Building [image1]...
  creating kaniko pod: pods is forbidden: User "system:serviceaccount:gitlab-runner:default" cannot create resource "pods" in API group "" in the namespace "gitlab-runner"