sregistry-cli with https

[tejasdixit836]

Hello,

I have trouble using sregistry-cli with sregistry portal, that i have set up with https

I am able to pull/push the images when i am running the sregistry with http but not with https

here is how i have configured https:

DOMAIN_NAME = "https://domainnname"
DOMAIN_NAME_HTTP = "http://domainname"
DOMAIN_NAKED = DOMAIN_NAME_HTTP.replace('http://', '')
(all the ssl certificates in place according to the document and the registry GUI looks fine)

when i am trying to pull/push using sregistry-cli i get this error

ERROR Issue with https://sregistry-domain/api/container/library/alpine:test, try exporting SREGISTRY_REGISTRY_NOHTTPS.

here is my .sregistry file

{
"hub": {
"base": "https://singularity-hub.org/api"
},
"registry": {
"SREGISTRY_REGISTRY_BASE": "https://domainname",
"SREGISTRY_REGISTRY_USERNAME": "user1",
"SREGISTRY_REGISTRY_TOKEN": "fe55533e17c865b33bf583f004fd3token"
},
"SREGISTRY_CLIENT": "registry"
}

if i replace "SREGISTRY_REGISTRY_BASE": "https://domainname" with "http" i get the following error

[client|registry] [database|sqlite:////home/user/.singularity/sregistry.db]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/contrib/pyopenssl.py", line 485, in wrap_socket
cnx.do_handshake()
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1934, in do_handshake
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1671, in _raise_ssl_error
_raise_current_error()
File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/connectionpool.py", line 662, in urlopen
self._prepare_proxy(conn)
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/connectionpool.py", line 948, in prepare_proxy
conn.connect()
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/connection.py", line 394, in connect
ssl_context=context,
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/util/ssl.py", line 370, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/contrib/pyopenssl.py", line 491, in wrap_socket
raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests-2.22.0-py3.6.egg/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.6/site-packages/urllib3-1.25.7-py3.6.egg/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='domainname', port=443): Max retries exceeded with url: /containers/2/download/e0f18dea-1bad-441d-ba31-7ce5e851faba (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/sregistry", line 11, in
load_entry_point('sregistry==0.2.32', 'console_scripts', 'sregistry')()
File "/usr/local/lib/python3.6/site-packages/sregistry-0.2.32-py3.6.egg/sregistry/client/init.py", line 323, in main
extra=extra)
File "/usr/local/lib/python3.6/site-packages/sregistry-0.2.32-py3.6.egg/sregistry/client/pull.py", line 37, in main
save=do_save)
File "/usr/local/lib/python3.6/site-packages/sregistry-0.2.32-py3.6.egg/sregistry/main/registry/pull.py", line 116, in pull
show_progress=not self.quiet)
File "/usr/local/lib/python3.6/site-packages/sregistry-0.2.32-py3.6.egg/sregistry/main/base/http.py", line 178, in download
if requests.head(url, verify=verify).status_code in [200, 401]:
File "/usr/local/lib/python3.6/site-packages/requests-2.22.0-py3.6.egg/requests/api.py", line 101, in head
return request('head', url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests-2.22.0-py3.6.egg/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests-2.22.0-py3.6.egg/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests-2.22.0-py3.6.egg/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests-2.22.0-py3.6.egg/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='domainname', port=443): Max retries exceeded with url: /containers/2/download/e0f18dea-1bad-441d-ba31-7ce5e851faba (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

can you please help

Thanks in advance

[vsoch]

Hmm, can you please first check the version of open ssl - see the two commands in this issue: https://stackoverflow.com/questions/50246084/django-paypalrestsdk-error-openssl-ssl-error-ssl-routines-tls-process-s

[vsoch]

Another thing to try is pushing/pulling with the Singularity client to see if that also produce an error.

[tejasdixit836]

Thanks for the quick response, singularity pull works fine, i am not able push however.

I add the endpoint as test

singularity remote add --no-login test container.page
singularity remote login test
INFO: Authenticating with remote: test
Generate an API Key at https://containerspage/auth/tokens, and paste here:
API Key:
FATAL: while verifying token: while getting token service uri: error response from server: 404

also an observation that the "usage" commands that gets generated in the portal (for example, if i have an image in my sregistry and i go to usage section to find the relevant commands to pull/push the images) they are not accurate. few of them work and few do not(my observation is that the subdirectory in collection gets skipped in the path) is there a way that i fix this? I am configuring this for users and they might use this as reference

what i get in container page: singularity pull shub://containerpage/milkshakes:banana
actaul pull which works: singularity pull shub://containerpage/milkshakes/pudding:banana

[vsoch]

Why are you using containers.page? Is that your domain? It’s just an example.

[vsoch]

If you pull with Singularity it should be a library:// uri so your testing urls are not correct.

[vsoch]

I’m going back to sleep :) Back in the morning!

[tejasdixit836]

openssl version is OpenSSL 1.0.2k-fips 26 Jan 2017

containerpage=domain
i am not sure about library, but
singularity pull shub://containerpage/milkshakes/pudding:banana is working fine.

sure, have a good sleep :) i will try to figure out something if i can mean while

[vsoch]

Good morning! So the first thing to figure out is why you aren't getting a response, period, with singularity remote login (404 means not found). If "https://container.page/" is really your domain, I don't see anything there, so there's where to start. You should minimally get another error if you have a correctly set up domain. Here is the documentation page for future reference in this issue.

[tejasdixit836]

it is indeed something to do with certificate, but i am not sure what is going wrong. I cannot put my domain name here so i gave container.page

it is interesting that singularity pull works with shub but not with library

singularity pull shub://domainname.com/newt/v1:latest
INFO: Downloading shub image
2.59 MiB / 2.59 MiB [============================================================================] 100.00% 287.67 KiB/s 9s

singularity pull --library https://hpcs-sregistry.shell.com newt/v1:latest
FATAL: While pulling library image: image newt/v1:latest (amd64) does not exist in the library

sregistry pull registry://newt/v1:latest
[client|registry] [database|sqlite:////glb/home/intdr3/.singularity/sregistry.db]
ERROR Issue with https://domainname/api/container/newt/v1:latest, try exporting SREGISTRY_REGISTRY_NOHTTPS.

singularity remote login test
INFO: Authenticating with remote: test
Generate an API Key at https://doaminname/auth/tokens, and paste here:
API Key:
FATAL: while verifying token: while getting token service uri: error response from server: 404

This is how i set up https,

• generated certs and placed it in /etc/ssl/certs and private
  certs has chained.pem and dhparam.pem
  private has domain.key
• then copied the docker-compose.yml from https folder in sregistry to sregistry folder
• started the docker-compose

the portal looks ok with https, push with sregistry works , but not with singularity.

when i in change .sregistry, the sregistry base from https://domainname to http://domainname then i get ssl verify error which i have posted yesterday in the begining

[tejasdixit836]

i could not reply sooner as i was trying to set up everyhting all over again which also dint help !!!

[vsoch]

An issue with certificates wouldn’t be helped by re-creating the server. I don’t know how else to help you without being able to reproduce, you likely haven’t set up your DNS if you can’t see the page in the browser and this is a downstream result of that.

[tejasdixit836]

i can see the page in the browser, i can see all the images that i pushed as well. just not able to use singularity client

[vsoch]

okay this is with https?

[tejasdixit836]

yes with https

[vsoch]

What version of Singularity?