Image Pull with SHA value from Local Registry is failing for some images.

[Sachin-Itagi]

Facing an issue with pulling mirrored images from the local registry using the SHA value.
The images were mirrored using the following command

skopeo copy --all --dest-tls-verify=false --dest-username=${PRIVATE_REGISTRY_PUSH_USER} --dest-password=${PRIVATE_REGISTRY_PUSH_PASSWORD} --preserve-digests docker://${SRC_REPO}/${SRC_IMG}@${SRC_DIGEST} docker://${PRIVATE_REGISTRY_LOCATION}/${SRC_IMG}:${SRC_TAG}

The following error is observed for some of the images.

podman pull registry.cp4d-414-dc0.192.168.0.23.nip.io:5000/cp/cpd/environments-ui@sha256:77b1791b9b52f3ec4504b93aa410593c9ce522d1140bcb3a57aa02f63f274dcf
Trying to pull registry.cp4d-414-dc0.192.168.0.23.nip.io:5000/cp/cpd/environments-ui@sha256:77b1791b9b52f3ec4504b93aa410593c9ce522d1140bcb3a57aa02f63f274dcf...Error: determining manifest MIME type for docker://registry.cp4d-414-dc0.192.168.0.23.nip.io:5000/cp/cpd/environments-ui@sha256:77b1791b9b52f3ec4504b93aa410593c9ce522d1140bcb3a57aa02f63f274dcf: Manifest does not match provided manifest digest sha256:77b1791b9b52f3ec4504b93aa410593c9ce522d1140bcb3a57aa02f63f274dcf

Had a look the manifest using the skopeo inspect command for the original and mirrored image and observed that the mirrored image has a different SHA value.

[root@sitagi-414-dc0-bastion-0 repositories]# skopeo inspect docker://cp.icr.io/cp/cpd/environments-api:8.3.0012-ppc64le | grep Digest
    "Digest": "sha256:ae3fc7524efe1f5eeb0dd18058af32fe50b4d156d10525a8d2d365f91c69f2a1",

[root@sitagi-414-dc0-bastion-0 repositories]# skopeo inspect docker://registry.sitagi-414-dc0.192.168.0.64.nip.io:5000/cp/cpd/environments-api:8.3.0012-ppc64le | grep Digest
    "Digest": "sha256:3aceab15422b076ecb00b12e5a193524788346cfb2f3da808e87e6a7ede48397",

Pulling the image using the different SHA of the mirrored image gives the following error.

[root@sitagi-414-dc0-bastion-0 repositories]# podman pull registry.sitagi-414-dc0.192.168.0.64.nip.io:5000/cp/cpd/environments-api@sha256:3aceab15422b076ecb00b12e5a193524788346cfb2f3da808e87e6a7ede48397
Trying to pull registry.sitagi-414-dc0.192.168.0.64.nip.io:5000/cp/cpd/environments-api@sha256:3aceab15422b076ecb00b12e5a193524788346cfb2f3da808e87e6a7ede48397...
Error: initializing source docker://registry.sitagi-414-dc0.192.168.0.64.nip.io:5000/cp/cpd/environments-api@sha256:3aceab15422b076ecb00b12e5a193524788346cfb2f3da808e87e6a7ede48397: reading manifest sha256:3aceab15422b076ecb00b12e5a193524788346cfb2f3da808e87e6a7ede48397 in registry.sitagi-414-dc0.192.168.0.64.nip.io:5000/cp/cpd/environments-api: manifest unknown

Issue Resolution
We replaced the local registry image from docker.io/ibmcom/registry-ppc64le:2.6.2.5 image with the official multi-arch image docker.io/library/registry:2 and this resolved the issue.
The image mirrored has the correct SHA value and image pull is working as expected.

Have raised a PR to replace the older image with the newer one. #518