Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pushing images to dockerhub stopped working #1209

Open
gebi opened this issue Apr 23, 2020 · 50 comments
Open

Pushing images to dockerhub stopped working #1209

gebi opened this issue Apr 23, 2020 · 50 comments
Labels
area/registry For all bugs having to do with pushing/pulling into registries gitlab kind/bug Something isn't working kind/friction priority/p1 Basic need feature compatibility with docker build. we should be working on this next. work-around-available

Comments

@gebi
Copy link

gebi commented Apr 23, 2020

Actual behavior

Kaniko exits with exit code 1 with the following message and and does not build the image:

error checking push permissions -- make sure you entered the correct tag name,
and that you are authenticated correctly, and try again: checking push permission for
"foo/bar:bionic-99": UNAUTHORIZED: authentication required; [map[Action:pull Class:
Name:mgit/clamav Type:repository] map[Action:push Class: Name:foo/bar Type:repository]]

This worked with the same build pipeline and no changes 3 months ago with the following image:

Using Docker executor with image gcr.io/kaniko-project/executor:debug ...
Pulling docker image gcr.io/kaniko-project/executor:debug ...
Using docker image sha256:2aa254b4837c242c7de87956438eaba70f97a2768ab0870819fd20e09df15cf6 for gcr.io/kaniko-project/executor:debug ...

Expected behavior

Kaniko to upload image to dockerhub like the version 3 months ago was able to.
There where no changes, and it works if i go back to an older kaniko version.

To Reproduce
Steps to reproduce the behavior:

  1. ... with the following pseudo .gitlab-ci.yml
image:
  name: gcr.io/kaniko-project/executor:debug
  entrypoint: [""]

stages:
  - foo

build-foo:
  stage: foo
  script:
    - echo "{\"auths\":{\"index.docker.io\":{\"auth\":\"${CI_DOCKERHUB_AUTH}\"}}}" > /kaniko/.docker/config.json
    - >
      /kaniko/executor --context "${CI_PROJECT_DIR}/foo" --dockerfile "${CI_PROJECT_DIR}/foo/Dockerfile"
      --destination foo/bar:blub-${CI_PIPELINE_IID}"
      --destination foo/bar:blub"
  1. ... build it

Additional Information

  • Dockerfile
    Please provide either the Dockerfile you're trying to build or one that can reproduce this error.
  • Build Context
    Please provide or clearly describe any files needed to build the Dockerfile (ADD/COPY commands)
  • Kaniko Image (fully qualified with digest)
Using Docker executor with image gcr.io/kaniko-project/executor:debug ...
Pulling docker image gcr.io/kaniko-project/executor:debug ...
Using docker image sha256:2ec307dcf7f52dcf700ea0fbc65d448f46365cfac69567e8177bf12b80942f54 for gcr.io/kaniko-project/executor:debug ...

Triage Notes for the Maintainers

Description Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile
@gebi
Copy link
Author

gebi commented Apr 23, 2020

Ah, sidenote, i confirmed that the credentials still work for pushing images to dockerhub, and they work fine, both manually and as mentioned with the old kaniko version.

For now we have pinned the kaniko version to this one mgit/base:kaniko-executor-debug-stable (which also fixes the problem of kaniko unable to build images on bigger FS because of the included busybox on bigger filesystems due to 64bit inodes and the included busybox not being compiled with large file support #656 )

@giovanism
Copy link
Contributor

Is this duplicate of #245?

@gebi
Copy link
Author

gebi commented Apr 30, 2020

I don't think so,
as in 245 they mention multistage builds and long build time. We have just a short build time, no multistage builds and it worked with the version of kaniko from a few months ago perfectly.

@Forst
Copy link

Forst commented May 5, 2020

Encountering this issue as well. The last working version seems to be v0.16.0.

Could #957 be causing the issue perhaps?

@gebi
Copy link
Author

gebi commented May 5, 2020

it seems #1005 describes the same problem with kaniko

@gebi
Copy link
Author

gebi commented May 5, 2020

i can verify that for us too the lastest working kaniko version is v0.16.0

v0.20.0 is not able to build the image, with the following job output:

$ mkdir -p /kaniko/.docker
$ echo "{\"auths\":{\"index.docker.io\":{\"auth\":\"${DOCKERHUB_AUTH}\"}}}" > /kaniko/.docker/config.json
$ mkdir /docker-tmp
$ echo 'FROM debian:stable' >> /docker-tmp/dockerfile
$ echo 'ENTRYPOINT ["/bin/bash", "-c", "echo hello"]' >> /docker-tmp/dockerfile
$ /kaniko/executor --context /docker-tmp --dockerfile /docker-tmp/dockerfile --destination foo/bar:hello-world-latest
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "foo/bar:hello-world-latest": POST https://index.docker.io/v2/foo/bar/blobs/uploads/: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:mgit/base Type:repository] map[Action:push Class: Name:mgit/base Type:repository]]Running after_script
Uploading artifacts for failed job
ERROR: Job failed: exit code 1
FATAL: exit code 1

@diegosasw
Copy link

diegosasw commented May 6, 2020

@gebi thanks, I confirm with the mgit/base:kaniko-executor-debug-stable image I was able to push to DockerHub but with gcr.io/kaniko-project/executor:debug I was experiencing the same error. Thanks!

This is my script

- export BASE64_CREDENTIALS="$(echo -n $DOCKER_HUB_REPOSITORY_USERNAME:$DOCKER_HUB_REPOSITORY_PASSWORD | base64)"
# @see https://github.com/GoogleContainerTools/kaniko/issues/1227 where folder may not exist
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"https://index.docker.io/v2/\":{\"auth\":\"$BASE64_CREDENTIALS\"}}}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/$IMAGE_TAG/Dockerfile --destination $IMAGE_NAME:$IMAGE_TAG

Used variables to make it more meaningful for newbies (like myself 1 hour ago)

@macrotex
Copy link

macrotex commented May 6, 2020

This is also happening to us using the lastest version of "debug" (May 6, 2020) attempting to push to GCR. Seeing this error:

E0506 17:32:28.676141      16 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Changing to tag debug-v0.19.0 things start working again.

brianseeders added a commit to brianseeders/kibana-ci-service that referenced this issue May 6, 2020
@tejal29
Copy link
Member

tejal29 commented May 7, 2020

@macrotex Can you please use kaniko v0.22.0 https://github.com/GoogleContainerTools/kaniko/releases/tag/v0.22.0 and let us know if it still exists.

@tejal29 tejal29 added area/registry For all bugs having to do with pushing/pulling into registries kind/bug Something isn't working priority/p1 Basic need feature compatibility with docker build. we should be working on this next. gitlab labels May 7, 2020
@macrotex
Copy link

macrotex commented May 8, 2020

This is also happening to us using the lastest version of "debug" (May 6, 2020) attempting to push to GCR. Seeing this error:

E0506 17:32:28.676141      16 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Changing to tag debug-v0.19.0 things start working again.

Version 0.22.0 fixed my issue.

@ymage
Copy link

ymage commented May 14, 2020

I tried to use https://index.docker.io/v1/ instead of v2 one as docker registry url, which seemed to work for me.
Is v2 endpoint really suported by kaniko ?

@GTB3NW
Copy link

GTB3NW commented May 15, 2020

Hey I still get the aws_credentials error:

Using Kubernetes executor with image gcr.io/kaniko-project/executor:debug-v0.22.0 ...

 E0515 00:08:48.365446      15 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

@anna-codefresh
Copy link

I too am getting the same error with v0.22.0. I tested this and the latest version to work is 0.16.0 as @gebi mentioned.

@ymage
Copy link

ymage commented May 18, 2020

This seems to work for docker hub :

$ export DOCKERHUB_AUTH="$(echo -n $DOCKER_HUB_REPOSITORY_USERNAME:$DOCKER_HUB_REPOSITORY_PASSWORD | base64)"
$ echo "{\"auths\":{\"https://index.docker.io/v1/\":{\"auth\":\"${DOCKERHUB_AUTH}\"}}}" > docker.json
$ docker run --rm -v $(pwd):/workspace -v $(pwd)/docker.json:/kaniko/.docker/config.json:ro gcr.io/kaniko-project/executor:v0.22.0 --context=dir:///workspace --dockerfile=Dockerfile --destination=foo/bar:latest

@anna-codefresh
Copy link

thanks @ymage. I was using the v2 docker endpoint instead of v1.

@vistaarjuneja
Copy link

+1. I am able to upload docker images with https://index.docker.io/v1 but not https://index.docker.io/v2 with the latest kaniko debug executor image. Is anyone working on this issue?

@anjeyy
Copy link

anjeyy commented Aug 11, 2020

And there i was sitting the last sunday half a day thinking to be that stupid to build a simple image which i wanted to push to my private docker hub.

auths: [https://index.docker.io/v2/]

did NOT work for me.

  • debug (without version)
  • debug-v0.22.0
  • debug-v0.19.0

did work for me.

  • debug-v0.16.0

auths: [https://index.docker.io/v1/]

did work for me.

  • debug (without version)
  • debug-v0.22.0
  • debug-v0.19.0
  • debug-v0.16.0

What combination should i use, since i have no idea what the difference it's making?

  • v1
  • v2

@livelace
Copy link

livelace commented Aug 11, 2020

works for me:

https://index.docker.io/v1/ + debug-v0.16.0

UP. It seems the real reason was UserAgent in config.json:

"HttpHeaders": {
  "User-Agent": "Docker-Client/19.03.12 (linux)"
}

After removing this section I haven't have any problems with pushing (even with original debug-539ddefcae3fd6b411a95982a830d987f4214251)

@tanguydelignieresaccenture
Copy link

tanguydelignieresaccenture commented Aug 25, 2020

After being struggled all the day with the issue, trying to push to dockerhub with a previous version of kaniko, debug-v0.18.0, which was fine few months ago in the same context, as @gebi, I was able to push the image using --registry-mirror index.docker.io.

@tejal29 may it be related to dockerhub hostname or default image path that has maybe changed and is no more compatible with kaniko (in old versions at least)?

I'm using auths with https://index.docker.io/v1/ in config.json.

KO:

$ docker run --rm --entrypoint "" -v /host/path/to/kaniko/config.json:/kaniko/.docker/config.json -v /host/path/to/dockerfile/directory/kaniko/20200825-001/build1:/workspace gcr.io/kaniko-project/executor:debug-v0.18.0 /kaniko/executor --context /workspace --dockerfile /workspace/Dockerfile --destination index.docker.io/tanguydelignieres/kaniko_bugs_20200825-001_build1:debug-v0.18.0
INFO[0003] Resolved base name alpine:3.9 to alpine:3.9
INFO[0003] Resolved base name alpine:3.9 to alpine:3.9
INFO[0003] Retrieving image manifest alpine:3.9
INFO[0005] Retrieving image manifest alpine:3.9
INFO[0009] Built cross stage deps: map[]
INFO[0009] Retrieving image manifest alpine:3.9
INFO[0011] Retrieving image manifest alpine:3.9
INFO[0014] Skipping unpacking as no commands require it.
INFO[0014] Taking snapshot of full filesystem...
INFO[0014] Resolving paths
INFO[0014] CMD echo "OK"
error pushing image: failed to push to destination index.docker.io/tanguydelignieres/kaniko_bugs_20200825-001_build1:debug-v0.18.0: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:tanguydelignieres/kaniko_bugs_20200825-001_build1 Type:repository] map[Action:push Class: Name:tanguydelignieres/kaniko_bugs_20200825-001_build1 Type:repository] map[Action:pull Class: Name:library/alpine Type:repository]]

$

OK with --registry-mirror index.docker.io:

$ docker run --rm --entrypoint "" -v /host/path/to/kaniko/config.json:/kaniko/.docker/config.json -v /host/path/to/dockerfile/directory/kaniko/20200825-001/build1:/workspace gcr.io/kaniko-project/executor:debug-v0.18.0 /kaniko/executor --registry-mirror index.docker.io --context /workspace --dockerfile /workspace/Dockerfile --destination index.docker.io/tanguydelignieres/kaniko_bugs_20200925-001_build1:debug-v0.18.0
INFO[0002] Resolved base name alpine:3.9 to alpine:3.9
INFO[0002] Resolved base name alpine:3.9 to alpine:3.9
INFO[0002] Retrieving image manifest alpine:3.9
INFO[0004] Retrieving image manifest alpine:3.9
INFO[0007] Built cross stage deps: map[]
INFO[0007] Retrieving image manifest alpine:3.9
INFO[0008] Retrieving image manifest alpine:3.9
INFO[0010] Skipping unpacking as no commands require it.
INFO[0010] Taking snapshot of full filesystem...
INFO[0010] Resolving paths
INFO[0010] CMD echo "OK"

$

I confirm I did not face the problem with debug-v1.0.0.

@nlamirault
Copy link

nlamirault commented Aug 27, 2020

I try with Kaniko v1.0.0 on Docker Hub v2 endpoint, it fails. It works using v1.

@tejal29
Copy link
Member

tejal29 commented Aug 27, 2020

Thank you folks, I updated the docs to use v1 endpoint instead of v2

@tejal29
Copy link
Member

tejal29 commented Aug 27, 2020

@tanguydelignieresaccenture i am still trying to understand why adding --registry-mirror works for you.

drGrove added a commit to drGrove/drone-kaniko that referenced this issue May 31, 2021
To get around GoogleContainerTools/kaniko#1209 we need to auth using the
DockerHub v1 API. As that variable is the "registry" and we have to use
a FQDN with Protocol we have to convert to standard registry format for
tagging the image, otherwise there's an issue with the auth mechanism
where it attempts to normalize the URL and incorrectly parses it.

GoogleContainerTools/kaniko#1209
drGrove added a commit to drGrove/drone-kaniko that referenced this issue Jun 1, 2021
To get around GoogleContainerTools/kaniko#1209 we need to auth using the
DockerHub v1 API. As that variable is the "registry" and we have to use
a FQDN with Protocol we have to convert to standard registry format for
tagging the image, otherwise there's an issue with the auth mechanism
where it attempts to normalize the URL and incorrectly parses it.

GoogleContainerTools/kaniko#1209
@qafro1
Copy link

qafro1 commented Jul 7, 2021

I'm having the same issue

ccwienk added a commit to gardener/cc-utils that referenced this issue Jul 26, 2021
Kaniko does (no longer?) seem to support authenticating against
"docker-hub"'s v2-api-endpoint, as also discussed on GitHub:

GoogleContainerTools/kaniko#1209

The suggested workaround seems to be to use the v1-endpoint. So hackily
inject it when generating a docker-cfg for kaniko.
@msolimans
Copy link

msolimans commented Aug 23, 2021

I used https://index.docker.io/v1/ and it worked

In k8s:

  • kubectl create secret docker-registry --docker-server=https://index.docker.io/v1/ --docker-username=x --docker-password=x
  • then use the secret anywhere you want

===============

using docker:

  • create docker.json file:
$ export DOCKERHUB_AUTH="$(echo -n userName:password | base64)"
$ echo "{\"auths\":{\"https://index.docker.io/v1/\":{\"auth\":\"${DOCKERHUB_AUTH}\"}}}" > docker.json
  • then
    docker run --rm -v $(pwd)/docker.json:/kaniko/.docker/config.json:ro gcr.io/kaniko-project/executor:debug --context=git://github.com/msolimans/icho --dockerfile=Dockerfile --destination=msolimans/icho:latest

@loarca
Copy link

loarca commented Sep 21, 2021

Ok sooo, since v1 is deprecated, I don't believe using v1 is the safe option here or is it?
I'm just confused. Isn't v1 supported only by older docker clients? (and thus meaning it would stop working someday)

Which makes me think that, from debug-v0.16.0 to debug-v0.19.0 something change that made the registries think kaniko is an old docker client? and thus blocking kaniko from pushing/pulling on v2? I don't know.

@svlentink
Copy link

svlentink commented Oct 16, 2021

After hours upon hours on this issue and keep getting the Unauthorized error, I did a docker login and found that the config looked as follows:

{
        "auths": {
                "https://index.docker.io/v1/": {
                        "auth": "REDACTED_T0="
                },
                "https://index.docker.io/v2/": {
                        "auth": "REDACTED_T0K"
                }
        }
}

When doing a base64 -d on both, it showed the same value. Thus the issue was that docker login uses a different base64 encode method for v1 and v2. (my password ended with an =)
When using Kaniko I removed the v2 auth and it now works.

@sichan-vonage
Copy link

We are using Harbor as a self hosted container registry, but I cannot get the authentification to work in the kaniko build. The error is the same as with the official Docker registry:

error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "hub.domain.com/repo/kaniko:f569d627": POST https://hub.domain.com/v2/repo/kaniko/blobs/uploads/: UNAUTHORIZED: unauthorized to access repository: repo/kaniko, action: push: unauthorized to access repository: repo/kaniko, action: push

I also tried the suggested workaround resulting in config.json like this:

{
        "auths": {
                "hub.domain.com/v1": {
                        "auth": "xxxxxxxxxxxx"
                }
}

But the logs/errors stayed the same, i.e. Kaniko still used the https://hub.domain.com/v2/ endpoint. Any hints how I could adapt the workaround to work with Harbor as well?

I'm having the exact same problem, did you ever find the fix to this?

@imjasonh
Copy link
Collaborator

imjasonh commented Apr 4, 2022

The index.docker.io key in the config file is special, for backward compatibility with DockerHub specifically. Even if the map key is https://index.docker.io/v1/, this is handled as a special case to mean "auth for index.docker.io, where registry API operations always speak to https://index.docker.io/v2/.../blobs/....

So if you're trying to auth to a Harbor registry at hub.domain.com, the map key should be "hub.domain.com", and not include either /v1 or /v2.

@sichan-vonage
Copy link

The index.docker.io key in the config file is special, for backward compatibility with DockerHub specifically. Even if the map key is https://index.docker.io/v1/, this is handled as a special case to mean "auth for index.docker.io, where registry API operations always speak to https://index.docker.io/v2/.../blobs/....

So if you're trying to auth to a Harbor registry at hub.domain.com, the map key should be "hub.domain.com", and not include either /v1 or /v2.

I've tried this without the /v1 or /v2 route and it still doesn't work..

In our setup, we are also using a service account to enable us to pull images from AWS ECR for our base image. But then our final built image is pushed to our own private registry. Is there any chance that using a combination of registries is preventing us from pushing to our custom registry?

@madrisan
Copy link

madrisan commented Apr 4, 2022

Have you set a proxy and forgot to no_proxy your local private registry?

@sichan-vonage
Copy link

Have you set a proxy and forgot to no_proxy your local private registry?

no, we have not set any proxies.

My config.json looks like this:

{
  "auths": {
    "private.registry.net": {
      "username": "admin",
      "password": "XXX",
      "auth": "YYY="
    }
  }
}

and I am setting kaniko destination value to private.registry.net/admin/client-app-poc:latest

The error I'm getting from kaniko is:

error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again:
checking push permission for "private.registry.net/admin/client-app-poc:latest": POST https://private.registry.net/v2/admin/client-app-poc/blobs/uploads/: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:admin/client-app-poc Type:repository] map[Action:push Class: Name:admin/client-app-poc Type:repository]]

@imjasonh
Copy link
Collaborator

imjasonh commented Apr 4, 2022

So first of all, it probably isn't the cause of the issue, but it would help resolve confusion to not use both username+password and auth; the auth field is meant to be the base64-encoded username+:+password, so just in case they don't match, we don't want any confusion about which one is used. I'd suggest removing the plaintext username+password fields, and just use the base64 auth field.

After removing those, make sure that docker push works -- this will tell us if the auth configured in the file is authorized to perform pushes. If that works and Kaniko doesn't, we have a bug and this debugging information will be really useful.

@sichan-vonage
Copy link

sichan-vonage commented Apr 4, 2022

it's okay I've finally sorted it, very daft mistake, but basically just had the config mounted in the wrong place...

Thank you guys for helping out!

@DarkDeddok
Copy link

it's okay I've finally sorted it, very daft mistake, but basically just had the config mounted in the wrong place...

Thank you guys for helping out!

you can provide part of the code where you mount?

@mohitbhansali
Copy link

We are using Harbor as a self hosted container registry, but I cannot get the authentification to work in the kaniko build. The error is the same as with the official Docker registry:

error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "hub.domain.com/repo/kaniko:f569d627": POST https://hub.domain.com/v2/repo/kaniko/blobs/uploads/: UNAUTHORIZED: unauthorized to access repository: repo/kaniko, action: push: unauthorized to access repository: repo/kaniko, action: push

I also tried the suggested workaround resulting in config.json like this:

{
        "auths": {
                "hub.domain.com/v1": {
                        "auth": "xxxxxxxxxxxx"
                }
}

But the logs/errors stayed the same, i.e. Kaniko still used the https://hub.domain.com/v2/ endpoint. Any hints how I could adapt the workaround to work with Harbor as well?

I'm having the exact same problem, did you ever find the fix to this?

Did you mange to resolve this issue?

@artemptushkin
Copy link

It's still not working with v2 but works with v1 in the config.json file, will this issue address this or we need/have another one?

@sgurusamy-rli
Copy link

are there any configuration changes required to make it work with docker api v2 (https://index.docker.io/v2?

@0xkevin
Copy link

0xkevin commented Oct 19, 2023

We are using Harbor as a self hosted container registry, but I cannot get the authentification to work in the kaniko build. The error is the same as with the official Docker registry:

error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "hub.domain.com/repo/kaniko:f569d627": POST https://hub.domain.com/v2/repo/kaniko/blobs/uploads/: UNAUTHORIZED: unauthorized to access repository: repo/kaniko, action: push: unauthorized to access repository: repo/kaniko, action: push

I also tried the suggested workaround resulting in config.json like this:

{
        "auths": {
                "hub.domain.com/v1": {
                        "auth": "xxxxxxxxxxxx"
                }
}

But the logs/errors stayed the same, i.e. Kaniko still used the https://hub.domain.com/v2/ endpoint. Any hints how I could adapt the workaround to work with Harbor as well?

I'm having the exact same problem, did you ever find the fix to this?

Did you mange to resolve this issue?

+1

@alexis974
Copy link

Just tested it today with https://index.docker.io/v2/ and it seems to work.

My gitlab-ci job:

deploy:
  stage: deploy
  image:
    name: gcr.io/kaniko-project/executor:v1.14.0-debug
    entrypoint: [""]
  script:
    - echo -n "{\"auths\":{\"https://index.docker.io/v2/\":{\"auth\":\"$(echo -n ${CI_REGISTRY_USER}:${CI_REGISTRY_PASSWORD} | base64)\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor
      --context "${CI_PROJECT_DIR}"
      --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
      --destination=docker.io/<user>/<img_name>:<my_tag>

@ilyabreev
Copy link

Still doesn't work on latest image with private gitlab registry.

docker run -ti --rm -v /code:/workspace -v `pwd`/config.json:/kaniko/.docker/config.json:ro gcr.io/kaniko-project/executor:debug  --dockerfile=./path/Dockerfile --destination=registry.gitlab.local/group/project:latest --insecure --insecure-registry registry.gitlab.local
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "registry.gitlab.local/group/project:latest": POST http://registry.gitlab.local/v2/group/project/blobs/uploads/: unexpected status code 404 Not Found:

followed by Gitlab 404 page

@mlec1
Copy link

mlec1 commented Aug 19, 2024

Still doesn't work on latest image with private gitlab registry.

Which version of Gitlab are you using ?? It works for me on a private Gitlab with gcr.io/kaniko-project/executor:v1.22.0-debug

@artemptushkin
Copy link

@mlec1 probably an old I used, it was long time ago for me

@xooooooooox
Copy link

Building an image with kaniko behind a proxy works for me. (see https://archives.docs.gitlab.com/16.11/ee/ci/docker/using_kaniko.html)

variables:
  CI_REGISTRY: "https://index.docker.io/v1/"
  CI_REGISTRY_USER: <your_dockerhub_username>
  CI_REGISTRY_PASSWORD: <your_dockerhub_password>
  CI_REGISTRY_IMAGE: '<docker_hub_username>/<xxxx>'
  http_proxy: <your-proxy>
  https_proxy: <your-proxy>
  no_proxy: <your-no-proxy>

stages:
  - build

build:
  stage: build
  tags:
    - k8s
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [ "" ]
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"},\"$(echo -n $CI_DEPENDENCY_PROXY_SERVER | awk -F[:] '{print $1}')\":{\"auth\":\"$(printf "%s:%s" ${CI_DEPENDENCY_PROXY_USER} "${CI_DEPENDENCY_PROXY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor
      --build-arg http_proxy="$http_proxy"
      --build-arg https_proxy="$https_proxy"
      --build-arg no_proxy="$no_proxy"
      --context "${CI_PROJECT_DIR}"
      --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
      --destination "$CI_REGISTRY_IMAGE:${CI_COMMIT_TAG}"
  rules:
    - if: $CI_COMMIT_TAG

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/registry For all bugs having to do with pushing/pulling into registries gitlab kind/bug Something isn't working kind/friction priority/p1 Basic need feature compatibility with docker build. we should be working on this next. work-around-available
Projects
None yet
Development

No branches or pull requests