fix: enable TLS based on URL scheme for sync extension

[evanebb]

What type of PR is this?
bug

Which issue does this PR fix:
#2557

What does this PR do / Why do we need it:

If an issue # is not available please add repro steps and logs showing the issue:

Testing done on this change:
Note: I haven't managed to run a full make yet, my Mac seems to not play nice with golangci-lint and starts leaking memory heavily :)

I spun up a local instance of zot (with locally built images) and an 'upstream' repository with HTTPS enabled using a self-signed certificate with the following docker-compose.yml:

services:
  zot:
    # image: localhost/project-zot/zot-linux-arm64:orig
    image: localhost/project-zot/zot-linux-arm64:fixed
    ports:
      - "8080:5000"
    volumes:
      - ./config.json:/etc/zot/config.json:ro
      - zot_data:/var/lib/registry

  registry:
    image: registry:2
    ports:
      - "127.0.0.1:443:443"
    volumes:
      - registry_data:/var/lib/registry
      - ./certs:/certs:ro
    environment:
      REGISTRY_HTTP_ADDR: 0.0.0.0:443
      REGISTRY_HTTP_TLS_CERTIFICATE: /certs/registry.crt
      REGISTRY_HTTP_TLS_KEY: /certs/registry.key

volumes:
  zot_data: {}
  registry_data: {}

And the corresponding config.json:

{
  "storage":{
    "rootDirectory": "/var/lib/registry",
    "gc": true
  },
  "http":{
    "address":"0.0.0.0",
    "port":"5000"
  },
  "log":{
    "level":"debug"
  },
  "extensions": {
    "search": {
      "enable": true
    },
    "ui": {
      "enable": true
    },
    "sync": {
      "enable": true,
      "registries": [
        {
          "urls": [
            "https://registry:443/"
          ],
          "content": [
            {
              "prefix": "**",
              "destination": "/docker-registry"
            }
          ],
          "onDemand": true,
          "tlsVerify": false,
          "onlySigned": false
        }
      ]
    }
  }
}

I pushed a test image (ubuntu:latest) into the upstream registry with:

docker run --network zot_default quay.io/skopeo/stable:latest copy docker://ubuntu:latest docker://registry:443/ubuntu:latest --tls-verify=false

And then tried pulling it through the zot instance on my local machine with:

docker pull localhost:8080/docker-registry/ubuntu:latest

When using a version of the zot image that doesn't have the fix applied, this results in the following error:

docker pull localhost:8080/docker-registry/ubuntu:latest
Error response from daemon: repository localhost:8080/docker-registry/ubuntu not found: name unknown: repository name not known to registry

The corresponding zot logs show the same thing:

{"level":"info","repository":"docker-registry/ubuntu","reference":"latest","goroutine":269,"caller":"zotregistry.dev/zot/pkg/api/routes.go:2023","time":"2024-10-26T13:52:31.549751129Z","message":"trying to get updated image by syncing on demand"}
{"level":"info","goroutine":249,"caller":"zotregistry.dev/zot/pkg/extensions/sync/service.go:508","time":"2024-10-26T13:52:31.549780254Z","message":"getting available client"}
{"level":"error","error":"Get \"https://registry:443/v2/\": tls: failed to verify certificate: x509: certificate signed by unknown authority","url":"https://registry:443/v2/","component":"sync","errorType":"*url.Error","goroutine":249,"caller":"zotregistry.dev/zot/pkg/extensions/sync/httpclient/client.go:272","time":"2024-10-26T13:52:31.566886838Z","message":"failed to make request"}
{"level":"error","error":"unable to ping any registry URLs","repository":"docker-registry/ubuntu","reference":"latest","goroutine":269,"caller":"zotregistry.dev/zot/pkg/api/routes.go:2027","time":"2024-10-26T13:52:31.566968713Z","message":"failed to sync image"}

If I build a local image with the fix and use that one, it seems to work fine:

docker pull localhost:8080/docker-registry/ubuntu:latest
latest: Pulling from docker-registry/ubuntu
1f6304731171: Pull complete
Digest: sha256:4f6ea18f656aa6acbec418aeafc4b99c49ed958a1a47b5cd8cafd22698454500
Status: Downloaded newer image for localhost:8080/docker-registry/ubuntu:latest
localhost:8080/docker-registry/ubuntu:latest

Once again, the corresponding zot logs:

{"level":"info","repository":"docker-registry/ubuntu","reference":"latest","goroutine":45,"caller":"zotregistry.dev/zot/pkg/api/routes.go:2023","time":"2024-10-26T14:03:00.949684962Z","message":"trying to get updated image by syncing on demand"}
{"level":"info","goroutine":47,"caller":"zotregistry.dev/zot/pkg/extensions/sync/service.go:508","time":"2024-10-26T14:03:00.949727254Z","message":"getting available client"}
{"level":"info","remote":"https://registry:443/","repository":"docker-registry/ubuntu","reference":"latest","goroutine":47,"caller":"zotregistry.dev/zot/pkg/extensions/sync/service.go:310","time":"2024-10-26T14:03:00.952089837Z","message":"syncing image"}
{"level":"info","remote image":"registry:443/ubuntu:latest","local image":"docker-registry/ubuntu:latest","goroutine":47,"caller":"zotregistry.dev/zot/pkg/extensions/sync/service.go:465","time":"2024-10-26T14:03:00.981154671Z","message":"syncing image"}
{"level":"info","syncTempDir":"/var/lib/registry/docker-registry/ubuntu/.sync/b61b0046-6ad4-491d-bc44-a3b79a739709/docker-registry/ubuntu","reference":"latest","goroutine":47,"caller":"zotregistry.dev/zot/pkg/extensions/sync/destination.go:100","time":"2024-10-26T14:03:01.087267296Z","message":"pushing synced local image to local registry"}
{"level":"warn","error":"cache miss","digest":"sha256:1f630473117188fe348ebdcf0da5e93138275af14007f15baf79967a365e647a","goroutine":47,"caller":"zotregistry.dev/zot/pkg/storage/imagestore/imagestore.go:1249","time":"2024-10-26T14:03:01.088820712Z","message":"not found in cache"}
{"level":"debug","src":"/var/lib/registry/docker-registry/ubuntu/.uploads/3effb8e1-0baf-4770-bd50-bf7daddc38e6","dstDigest":"sha256:1f630473117188fe348ebdcf0da5e93138275af14007f15baf79967a365e647a","dst":"/var/lib/registry/docker-registry/ubuntu/blobs/sha256/1f630473117188fe348ebdcf0da5e93138275af14007f15baf79967a365e647a","goroutine":47,"caller":"zotregistry.dev/zot/pkg/storage/imagestore/imagestore.go:1043","time":"2024-10-26T14:03:01.115857212Z","message":"dedupe begin"}
{"level":"debug","src":"/var/lib/registry/docker-registry/ubuntu/.uploads/3effb8e1-0baf-4770-bd50-bf7daddc38e6","dst":"/var/lib/registry/docker-registry/ubuntu/blobs/sha256/1f630473117188fe348ebdcf0da5e93138275af14007f15baf79967a365e647a","component":"dedupe","goroutine":47,"caller":"zotregistry.dev/zot/pkg/storage/imagestore/imagestore.go:1069","time":"2024-10-26T14:03:01.117703462Z","message":"rename"}
{"level":"warn","error":"cache miss","digest":"sha256:f825f99d3d8a5d72586ed4d02ba949a7235e37effd735e7e258b5667ec805834","goroutine":47,"caller":"zotregistry.dev/zot/pkg/storage/imagestore/imagestore.go:1249","time":"2024-10-26T14:03:01.119457504Z","message":"not found in cache"}
{"level":"debug","src":"/var/lib/registry/docker-registry/ubuntu/.uploads/97c7e868-196d-4d31-996a-54641f36c812","dstDigest":"sha256:f825f99d3d8a5d72586ed4d02ba949a7235e37effd735e7e258b5667ec805834","dst":"/var/lib/registry/docker-registry/ubuntu/blobs/sha256/f825f99d3d8a5d72586ed4d02ba949a7235e37effd735e7e258b5667ec805834","goroutine":47,"caller":"zotregistry.dev/zot/pkg/storage/imagestore/imagestore.go:1043","time":"2024-10-26T14:03:01.119738004Z","message":"dedupe begin"}
{"level":"debug","src":"/var/lib/registry/docker-registry/ubuntu/.uploads/97c7e868-196d-4d31-996a-54641f36c812","dst":"/var/lib/registry/docker-registry/ubuntu/blobs/sha256/f825f99d3d8a5d72586ed4d02ba949a7235e37effd735e7e258b5667ec805834","component":"dedupe","goroutine":47,"caller":"zotregistry.dev/zot/pkg/storage/imagestore/imagestore.go:1069","time":"2024-10-26T14:03:01.132801087Z","message":"rename"}
{"level":"debug","repo":"docker-registry/ubuntu","reference":"latest","goroutine":47,"caller":"zotregistry.dev/zot/pkg/extensions/sync/destination.go:247","time":"2024-10-26T14:03:01.136844129Z","message":"successfully set metadata for image"}
{"level":"info","image":"docker-registry/ubuntu:latest","goroutine":47,"caller":"zotregistry.dev/zot/pkg/extensions/sync/destination.go:184","time":"2024-10-26T14:03:01.136868462Z","message":"successfully synced image"}
{"level":"info","component":"sync","image":"registry:443/ubuntu:latest","goroutine":47,"caller":"zotregistry.dev/zot/pkg/extensions/sync/service.go:496","time":"2024-10-26T14:03:01.138475962Z","message":"finished syncing image"}

Automation added to e2e:
I haven't added anything, if it is desired for this change please let me know.

Will this break upgrades or downgrades?
Not as far as I know.
Does this PR introduce any user-facing change?:
No.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[rchincha]

@evanebb thanks for your PR. Started a CI run, let's see what happens.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.08%. Comparing base (51e779f) to head (d441f35).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2747   +/-   ##
=======================================
  Coverage   92.07%   92.08%           
=======================================
  Files         169      169           
  Lines       30052    30054    +2     
=======================================
+ Hits        27671    27674    +3     
+ Misses       1762     1760    -2     
- Partials      619      620    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[andaaron]

LGTM. @eusebiu-constantin-petu-dbk, can you please take a look too?

[rchincha]

@evanebb

git commit --amend --no-edit -s -S

and push again. This should resolve the DCO failure.

The PR looks good btw.

[eusebiu-constantin-petu-dbk]

lgtm and thank you for fixing it.

[evanebb]

Thank you! I amended the commit to add the DCO and force pushed it, so let's see if the workflow is happy now.