CURL_CA_BUNDLE= disables certificate verification

[owtaylor]

I'm not the first to notice this, see:

https://stackoverflow.com/questions/48391750/disable-python-requests-ssl-validation-for-an-imported-module

Which implies people have even relied on the current behavior as a hack ... but I think it's pretty clear that the current behavior is an accidental bug, which should be fixed (for requests 3?)

Vaguely related to #3829

Expected Result

An empty-string CURL_CA_BUNDLE should use default system verification, the same way as:

• An unset CURL_CA_BUNDLE
• An empty-string or unset REQUESTS_CA_BUNDLE
• Behavior of curl/libcurl with an empty-string or unset CURL_CA_BUNDLE

Actual Result

Empty CURL_CA_BUNDLE disables certificate verification

Reproduction Steps

• Set CURL_CA_BUNDLE to an empty value, try to fetch a self-signed or invalid HTTPS endpoint => success

[nateprewitt]

Hi @owtaylor,

Thanks for bringing this to our attention! This was definitely not intended behavior and we agree it's a bug. We've pushed up a PR (#6074) to fix the issue in the next minor release of Requests. While this may be breaking for the workflow discussed in Stackoverflow, that was never a supported method for disabling verification.

Please let us know if you have any other concerns or comments on the PR. Thanks!

[moshekaplan]

@nateprewitt : It pains me to request this, but given how popular the hack is, do you think it might make sense to create an environment variable for globally disabling SSL validation in requests?

[sigmavirus24]

given how popular the hack is,

Can you give evidence of this being popular?

[moshekaplan]

given how popular the hack is,

Can you give evidence of this being popular?

https://stackoverflow.com/questions/48391750/disable-python-requests-ssl-validation-for-an-imported-module has approximately 50 upvotes and 29k views.

[sigmavirus24]

given how popular the hack is,

Can you give evidence of this being popular?

https://stackoverflow.com/questions/48391750/disable-python-requests-ssl-validation-for-an-imported-module has approximately 50 upvotes and 29k views.

Yeah, we're not interested in giving people a global hammer to misuse based off of view numbers that don't reflect how widely used that was. I have viewed stack overflow answers that were useless because a search engine thought them relevant. You're qualification for popular is deeply flawed

[nateprewitt]

do you think it might make sense to create an environment variable for globally disabling SSL validation in requests?

While we take breakages very seriously in Requests due its wide usage, I think this falls far outside the realm of intended behavior. I don't believe we have interest in adding an alternative environment variable either. Requests has historically limited usage of env vars because of their "magic" behavior between systems.

The main concern is once this is set, it's very easy to forget, leading to unintended consequences. Globally disabling cert verification not only affects direct Requests invocations but any tool using Requests, which is not something we want to support.

If tools using Requests are working against endpoints without an accessible cert bundle, they should either expose a configurable endpoint or method to disable verification which is then passed to requests verify argument.

[jaklan]

@nateprewitt they should either expose a configurable endpoint or method to disable verification which is then passed to requests verify argument - maybe they should, but they don't always do, so having any option to disable the verification after removing the above hack would be quite important. Cannot verify be also configured with some envar like REQUESTS_SSL_VERIFY=[true|false]?

[sigmavirus24]

It can not, no. We don't support that. We shouldn't support that. To quote myself:

Yeah, we're not interested in giving people a global hammer

And quoting Nate,

The main concern is once this is set, it's very easy to forget, leading to unintended consequences. Globally disabling cert verification not only affects direct Requests invocations but any tool using Requests, which is not something we want to support.

This isn't going to happen.

[jaklan]

@sigmavirus24 / @nateprewitt - so what would be your "unofficial" recommendation to disable SSL verification for requests calls in 3rd party libraries (other than waiting for maintainers to address the issue / forking the repo to add verify=False)? Ofc the best solution would be to provide custom CA_BUNDLE and pass the verification, but sometimes it's problematic to figure out the proper one if the internal networking setup is messy, and you need a way to bypass that in the meantime to allow people do their job.

[sigmavirus24]

you need a way to bypass that in the meantime to allow people do their job.

Pin/Cap the version of Requests you use and use the old bypass. We can't and won't support 100% of possible users. We have to do the best to secure as many users as possible and prevent them from accidentally injuring themselves with a large and dangerous weapon.

[sigmavirus24]

Also if your jobs are so dependent upon this software, your employers should be sponsoring the PSF

[jaklan]

@sigmavirus24 I didn't mean to convince you to implement the workaround, I see your point, I just wanted to ask if there were some non-obvious, probably no less dirty than the old one, workarounds which can be used in such situation which you were aware of.

I heard about PYTHONHTTPSVERIFY=0, but it seems it doesn't have any effect on requests, right?

Also if your jobs are so dependent upon this software, your employers should be sponsoring the PSF

If developers had impact on corporate budgets, especially in non-IT-first companies... Book-size topic.