`OpenSSL error` breaking nimble and every package

[timotheecour]

yet another justification for the need for nimble-wide CI (PR #10247): a recent commit broke a number of packages on osx

• see https://travis-ci.org/genotrance/nimterop/jobs/478487797 reported by @genotrance

• likewise, running nimble test from a fresh clone of nimble (using latest nim devel) fails:

./nimble test
   Warning: Using env var NIM_LIB_PREFIX: /Users/timothee/git_clone//nim//Nim/
  Executing task test in /Users/timothee/git_clone/nim/nimble/nimble.nimble
Hint: used config file '/Users/timothee/git_clone/nim/Nim/config/nim.cfg' [Conf]
Hint: used config file '/Users/timothee/.config/nim/nim.cfg' [Conf]
Hint: used config file '/Users/timothee/git_clone/nim/Nim/config/config.nims' [Conf]
Hint:  [Link]
Hint: operation successful (66232 lines compiled; 2.207 sec total; 114.164MiB peakmem; Debug Build) [SuccessX]
Hint: /Users/timothee/git_clone/nim/nimble/tests/tester  [Exec]
[OK] can compile with --os:windows
    Reading config file at /Users/timothee/.config/nimble/nimble.ini
   Warning: Using env var NIM_LIB_PREFIX: /Users/timothee/git_clone//nim//Nim/
    Setting Nim stdlib prefix to /Users/timothee/git_clone//nim//Nim/
    Setting Nim stdlib path to /Users/timothee/git_clone/nim/Nim/lib
       Info Hint: used config file '/Users/timothee/git_clone/nim/Nim/config/nim.cfg' [Conf]
  Verifying dependencies for aporiaScenario@0.1.0
    Reading official package list
    Prompt: No local packages.json found, download it from internet? -> [forced yes]
Downloading official package list
     Trying http://google.com
   Warning: Downloaded packages.json file is invalid, discarding.
     Trying http://google.com/404
   Warning: Could not download: 404 Not Found
     Trying http://irclogs.nim-lang.org/packages.json
   Warning: Could not download: Please upgrade your OpenSSL library, it does not support the necessary protocols. OpenSSL error is: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
     Trying http://nim-lang.org/nimble/packages.json
     Trying https://github.com/nim-lang/packages/raw/master/packages.json
   Warning: Could not download: Please upgrade your OpenSSL library, it does not support the necessary protocols. OpenSSL error is: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
     Error: /Users/timothee/git_clone/nim/nimble/src/nimble.nim(1118) nimble
        ... /Users/timothee/git_clone/nim/nimble/src/nimble.nim(1056) doAction
        ... /Users/timothee/git_clone/nim/nimble/src/nimble.nim(474) install
        ... /Users/timothee/git_clone/nim/nimble/src/nimble.nim(346) installFromDir
        ... /Users/timothee/git_clone/nim/nimble/src/nimble.nim(168) processDeps
        ... /Users/timothee/git_clone/nim/nimble/src/nimblepkg/packageinfo.nim(369) resolveAlias
        ... /Users/timothee/git_clone/nim/nimble/src/nimblepkg/packageinfo.nim(271) getPackage
        ... /Users/timothee/git_clone/nim/nimble/src/nimblepkg/packageinfo.nim(241) readPackageList
        ... /Users/timothee/git_clone/nim/nimble/src/nimblepkg/packageinfo.nim(229) fetchList
        ... Refresh failed
        ... Could not download: Please upgrade your OpenSSL library, it does not support the necessary protocols. OpenSSL error is: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

Note:

my openssl is already up to date though
brew upgrade openssl
Error: openssl 1.0.2q already installed

potential cause

#10230 /cc @treeform @alaviss

[timotheecour]

brew info openssl
openssl: stable 1.0.2q [keg-only]
SSL/TLS cryptography library
https://openssl.org/
/Users/timothee/homebrew/Cellar/openssl/1.0.2l (1,701 files, 12.2MB)
  Built from source on 2017-09-03 at 21:04:49
/Users/timothee/homebrew/Cellar/openssl/1.0.2m (1,784 files, 12.3MB)
  Built from source on 2017-11-26 at 15:54:24
/Users/timothee/homebrew/Cellar/openssl/1.0.2n (1,784 files, 12.3MB)
  Built from source on 2017-12-12 at 22:21:46
/Users/timothee/homebrew/Cellar/openssl/1.0.2o_1 (1,783 files, 12.3MB)
  Built from source on 2018-03-31 at 17:01:57
/Users/timothee/homebrew/Cellar/openssl/1.0.2o_2 (1,784 files, 12.3MB)
  Built from source on 2018-06-19 at 22:30:26
/Users/timothee/homebrew/Cellar/openssl/1.0.2p (1,785 files, 12.3MB)
  Built from source on 2018-08-23 at 01:30:39
/Users/timothee/homebrew/Cellar/openssl/1.0.2q (1,786 files, 12MB)
  Built from source on 2018-11-29 at 12:17:36
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/openssl.rb
==> Caveats
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /Users/timothee/homebrew/etc/openssl/certs

and run
  /Users/timothee/homebrew/opt/openssl/bin/c_rehash

openssl is keg-only, which means it was not symlinked into /Users/timothee/homebrew,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

If you need to have openssl first in your PATH run:
  echo 'export PATH="/Users/timothee/homebrew/opt/openssl/bin:$PATH"' >> ~/.zshrc

For compilers to find openssl you may need to set:
  export LDFLAGS="-L/Users/timothee/homebrew/opt/openssl/lib"
  export CPPFLAGS="-I/Users/timothee/homebrew/opt/openssl/include"

For pkg-config to find openssl you may need to set:
  export PKG_CONFIG_PATH="/Users/timothee/homebrew/opt/openssl/lib/pkgconfig"

==> Analytics
install: 473,391 (30 days), 1,502,254 (90 days), 6,015,040 (365 days)
install_on_request: 66,055 (30 days), 210,830 (90 days), 840,903 (365 days)
build_error: 0 (30 days)

[treeform]

I had my OpenSSL problems on OSX only. I think looking for OpenSSL in random magical order that happens to work is silly. Its very machine depended and hard to debug.

Which ssl version does it load for you?
Which ssl version does it load if you search it in correct order from new to old?

I think the code might claim that it works with a some version of openSSL when it does not. We need to figure out which version is that.

The revert probably makes it work by finding an older version of openSSL first which nim supports.

[timotheecour]

@treeform regression fixes always take precedence over bug fixes; #10230 might've fixed a bug but it introduced a regression, so it made sense to revert until we find a proper fix, and I only reverted the part that was problematic, OSX; I agree the existing code needs to be fixed properly, but priority was to fix the regression first.

Which ssl version does it load for you?

unfortunately otool -L doesn't work for nim binaries because of (#9203 or nim-lang/RFCs#58)

I/we can investigate this now that regression was fixed

[timotheecour]

reopened #9419 , so I'm removing the TODO

[Araq]

unfortunately otool -L doesn't work for nim binaries because of (#9203 or nim-lang/RFCs#58)

Oh not that again, so use OSX's strace variant.