Skip to content

explicit asking to get cacert.pem from certifi#145

Closed
prabcs wants to merge 2 commits intojohn-kurkowski:masterfrom
prabcs:upgrade_for_new_requests_version
Closed

explicit asking to get cacert.pem from certifi#145
prabcs wants to merge 2 commits intojohn-kurkowski:masterfrom
prabcs:upgrade_for_new_requests_version

Conversation

@prabcs
Copy link

@prabcs prabcs commented Mar 2, 2018

This PR intends to provide the ability to the session to find the cacert.pem file required by requests. It makes find_first_response work in the case when tldextract, requests, certifi are all bundled in one .egg/.zip file and the code is run from that bundle leading to this error -

File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/tldextract/tldextract.py", line 329, in extract
    return TLD_EXTRACTOR(url)
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/tldextract/tldextract.py", line 209, in __call__
    suffix_index = self._get_tld_extractor().suffix_index(translations)
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/tldextract/tldextract.py", line 249, in _get_tld_extractor
    raw_suffix_list_data = find_first_response(self.suffix_list_urls)
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/tldextract/remote.py", line 38, in find_first_response
    text = session.get(url).text
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/requests/adapters.py", line 407, in send
    self.cert_verify(conn, request.url, verify, cert)
  File "./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/requests/adapters.py", line 226, in cert_verify
    "invalid path: {0}".format(cert_loc))
IOError: Could not find a suitable TLS CA certificate bundle, invalid path: ./packages-e79cd9628c5bca67379a4f131d342e8420480100.egg/certifi/cacert.pem

@prabcs prabcs force-pushed the upgrade_for_new_requests_version branch from a9c0174 to 426f8cc Compare March 2, 2018 20:20
prabcs added a commit to Shopify/pyreferrer that referenced this pull request Mar 2, 2018
@prabcs
Copy link
Author

prabcs commented Mar 3, 2018

Requesting review - @john-kurkowski . Thanks !

@floer32
Copy link
Collaborator

floer32 commented Mar 4, 2019

@prabcs thanks!

need a little more info

you mentioned that this happens in a case when things are installed a certain way. when does that case happen in practice, exactly?

trying to understand because, if we pin certifi to a higher version, it could impact other dependency resolutions ... don't want to break somebody who is using an older requests, if that requests still works for us. (it could be tested to confirm/deny if it is an issue ...) (and if it is an issue, it could just be a 'next major version' upgrade - which could be soon, just need to know)

@floer32 floer32 added the icebox: needs clarification OP, please clarify (or someone else who desires the change) label Mar 4, 2019
@john-kurkowski
Copy link
Owner

Code looks good. Per the previous comment, it's good to consider the dependency ramifications, and how unique this case is.

@floer32
Copy link
Collaborator

floer32 commented Mar 5, 2019

@prabcs @john-kurkowski ahh here's the resolution. The packaging/dependency ramifications can be avoided completely. Instead of setting the verify argument for every get() (etc), it can be set on the session session_instance.verify = <your argument> -- see docs, http://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification

so in your case you'd want the session to have session.verify = resource_filename('certifi', 'cacert.pem'). but that's a detail that is ideal to put in your calling code, not in the main tldextract library -- separation of concerns/ open ended for downstream users

There is another PR where the same change (i.e. allow passing in a custom session instance) solves that problem too. #150 so these are going to be 2 birds 1 stone

@floer32
Copy link
Collaborator

floer32 commented Mar 5, 2019

created issue #158 to track that. closing this PR because #158 now exists, and because PR #150 is slightly closer to merge-able - i.e. that one has some tests. OP, feel free to do a new branch, or to go on top of PR #150. but let others know (on #150) if you pick it up.

@floer32 floer32 added see issue #158 [one fell swoop] fixed when we fix https://github.com/john-kurkowski/tldextract/issues/158 - temporary label. and removed icebox: needs clarification OP, please clarify (or someone else who desires the change) labels Mar 5, 2019
@floer32 floer32 closed this Mar 5, 2019
Guangyang-Sunlight added a commit to Guangyang-Sunlight/pyreferrer that referenced this pull request Jun 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

see issue #158 [one fell swoop] fixed when we fix https://github.com/john-kurkowski/tldextract/issues/158 - temporary label.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants